1APACHECOUCHDB(1) Apache CouchDB® APACHECOUCHDB(1)
2
6 apachecouchdb - Apache CouchDB® 3.0.0
7
9 CouchDB is a database that completely embraces the web. Store your data
10 with JSON documents. Access your documents with your web browser, via
11 HTTP. Query, combine, and transform your documents with JavaScript.
12 CouchDB works well with modern web and mobile apps. You can distribute
13 your data, efficiently using CouchDB’s incremental replication. CouchDB
14 supports master-master setups with automatic conflict detection.
15 CouchDB comes with a suite of features, such as on-the-fly document
17 transformation and real-time change notifications, that make web devel‐
18 opment a breeze. It even comes with an easy to use web administration
19 console, served directly out of CouchDB! We care a lot about
20 distributed scaling. CouchDB is highly available and partition toler‐
21 ant, but is also eventually consistent. And we care a lot about your
22 data. CouchDB has a fault-tolerant storage engine that puts the safety
23 of your data first.
24 In this section you’ll learn about every basic bit of CouchDB, see upon
26 what conceptions and technologies it built and walk through short tuto‐
27 rial that teach how to use CouchDB.
28 Technical Overview
30 Document Storage
31 A CouchDB server hosts named databases, which store documents. Each
32 document is uniquely named in the database, and CouchDB provides a
33 RESTful HTTP API for reading and updating (add, edit, delete) database
34 documents.
35 Documents are the primary unit of data in CouchDB and consist of any
37 number of fields and attachments. Documents also include metadata
38 that’s maintained by the database system. Document fields are uniquely
39 named and contain values of varying types (text, number, boolean,
40 lists, etc), and there is no set limit to text size or element count.
41 The CouchDB document update model is lockless and optimistic. Document
43 edits are made by client applications loading documents, applying
44 changes, and saving them back to the database. If another client edit‐
45 ing the same document saves their changes first, the client gets an
46 edit conflict error on save. To resolve the update conflict, the latest
47 document version can be opened, the edits reapplied and the update
48 tried again.
49 Single document updates (add, edit, delete) are all or nothing, either
51 succeeding entirely or failing completely. The database never contains
52 partially saved or edited documents.
53 ACID Properties
55 The CouchDB file layout and commitment system features all Atomic Con‐
56 sistent Isolated Durable (ACID) properties. On-disk, CouchDB never
57 overwrites committed data or associated structures, ensuring the data‐
58 base file is always in a consistent state. This is a “crash-only”
59 design where the CouchDB server does not go through a shut down
60 process, it’s simply terminated.
61 Document updates (add, edit, delete) are serialized, except for binary
63 blobs which are written concurrently. Database readers are never locked
64 out and never have to wait on writers or other readers. Any number of
65 clients can be reading documents without being locked out or inter‐
66 rupted by concurrent updates, even on the same document. CouchDB read
67 operations use a Multi-Version Concurrency Control (MVCC) model where
68 each client sees a consistent snapshot of the database from the begin‐
69 ning to the end of the read operation. This means that CouchDB can
70 guarantee transactional semantics on a per-document basis.
71 Documents are indexed in B-trees by their name (DocID) and a Sequence
73 ID. Each update to a database instance generates a new sequential num‐
74 ber. Sequence IDs are used later for incrementally finding changes in
75 a database. These B-tree indexes are updated simultaneously when docu‐
76 ments are saved or deleted. The index updates always occur at the end
77 of the file (append-only updates).
78 Documents have the advantage of data being already conveniently pack‐
80 aged for storage rather than split out across numerous tables and rows
81 in most database systems. When documents are committed to disk, the
82 document fields and metadata are packed into buffers, sequentially one
83 document after another (helpful later for efficient building of views).
84 When CouchDB documents are updated, all data and associated indexes are
86 flushed to disk and the transactional commit always leaves the database
87 in a completely consistent state. Commits occur in two steps:
88 1. All document data and associated index updates are synchronously
90 flushed to disk.
91 2. The updated database header is written in two consecutive, identical
93 chunks to make up the first 4k of the file, and then synchronously
94 flushed to disk.
95 In the event of an OS crash or power failure during step 1, the par‐
97 tially flushed updates are simply forgotten on restart. If such a crash
98 happens during step 2 (committing the header), a surviving copy of the
99 previous identical headers will remain, ensuring coherency of all pre‐
100 viously committed data. Excepting the header area, consistency checks
101 or fix-ups after a crash or a power failure are never necessary.
102 Compaction
104 Wasted space is recovered by occasional compaction. On schedule, or
105 when the database file exceeds a certain amount of wasted space, the
106 compaction process clones all the active data to a new file and then
107 discards the old file. The database remains completely online the
108 entire time and all updates and reads are allowed to complete success‐
109 fully. The old database file is deleted only when all the data has been
110 copied and all users transitioned to the new file.
111 Views
113 ACID properties only deal with storage and updates, but we also need
114 the ability to show our data in interesting and useful ways. Unlike SQL
115 databases where data must be carefully decomposed into tables, data in
116 CouchDB is stored in semi-structured documents. CouchDB documents are
117 flexible and each has its own implicit structure, which alleviates the
118 most difficult problems and pitfalls of bi-directionally replicating
119 table schemas and their contained data.
120 But beyond acting as a fancy file server, a simple document model for
122 data storage and sharing is too simple to build real applications on –
123 it simply doesn’t do enough of the things we want and expect. We want
124 to slice and dice and see our data in many different ways. What is
125 needed is a way to filter, organize and report on data that hasn’t been
126 decomposed into tables.
127 SEE ALSO:
129 views
130 View Model
132 To address this problem of adding structure back to unstructured and
133 semi-structured data, CouchDB integrates a view model. Views are the
134 method of aggregating and reporting on the documents in a database, and
135 are built on-demand to aggregate, join and report on database docu‐
136 ments. Because views are built dynamically and don’t affect the under‐
137 lying document, you can have as many different view representations of
138 the same data as you like.
139 View definitions are strictly virtual and only display the documents
141 from the current database instance, making them separate from the data
142 they display and compatible with replication. CouchDB views are defined
143 inside special design documents and can replicate across database
144 instances like regular documents, so that not only data replicates in
145 CouchDB, but entire application designs replicate too.
146 JavaScript View Functions
148 Views are defined using JavaScript functions acting as the map part in
149 a map-reduce system. A view function takes a CouchDB document as an
150 argument and then does whatever computation it needs to do to determine
151 the data that is to be made available through the view, if any. It can
152 add multiple rows to the view based on a single document, or it can add
153 no rows at all.
154 SEE ALSO:
156 viewfun
157 View Indexes
159 Views are a dynamic representation of the actual document contents of a
160 database, and CouchDB makes it easy to create useful views of data.
161 But generating a view of a database with hundreds of thousands or mil‐
162 lions of documents is time and resource consuming, it’s not something
163 the system should do from scratch each time.
164 To keep view querying fast, the view engine maintains indexes of its
166 views, and incrementally updates them to reflect changes in the data‐
167 base. CouchDB’s core design is largely optimized around the need for
168 efficient, incremental creation of views and their indexes.
169 Views and their functions are defined inside special “design” docu‐
171 ments, and a design document may contain any number of uniquely named
172 view functions. When a user opens a view and its index is automati‐
173 cally updated, all the views in the same design document are indexed as
174 a single group.
175 The view builder uses the database sequence ID to determine if the view
177 group is fully up-to-date with the database. If not, the view engine
178 examines all database documents (in packed sequential order) changed
179 since the last refresh. Documents are read in the order they occur in
180 the disk file, reducing the frequency and cost of disk head seeks.
181 The views can be read and queried simultaneously while also being
183 refreshed. If a client is slowly streaming out the contents of a large
184 view, the same view can be concurrently opened and refreshed for
185 another client without blocking the first client. This is true for any
186 number of simultaneous client readers, who can read and query the view
187 while the index is concurrently being refreshed for other clients with‐
188 out causing problems for the readers.
189 As documents are processed by the view engine through your ‘map’ and
191 ‘reduce’ functions, their previous row values are removed from the view
192 indexes, if they exist. If the document is selected by a view function,
193 the function results are inserted into the view as a new row.
194 When view index changes are written to disk, the updates are always
196 appended at the end of the file, serving to both reduce disk head seek
197 times during disk commits and to ensure crashes and power failures can
198 not cause corruption of indexes. If a crash occurs while updating a
199 view index, the incomplete index updates are simply lost and rebuilt
200 incrementally from its previously committed state.
201 Security and Validation
203 To protect who can read and update documents, CouchDB has a simple
204 reader access and update validation model that can be extended to
205 implement custom security models.
206 SEE ALSO:
208 api/db/security
209 Administrator Access
211 CouchDB database instances have administrator accounts. Administrator
212 accounts can create other administrator accounts and update design doc‐
213 uments. Design documents are special documents containing view defini‐
214 tions and other special formulas, as well as regular fields and blobs.
215 Update Validation
217 As documents are written to disk, they can be validated dynamically by
218 JavaScript functions for both security and data validation. When the
219 document passes all the formula validation criteria, the update is
220 allowed to continue. If the validation fails, the update is aborted
221 and the user client gets an error response.
222 Both the user’s credentials and the updated document are given as
224 inputs to the validation formula, and can be used to implement custom
225 security models by validating a user’s permissions to update a docu‐
226 ment.
227 A basic “author only” update document model is trivial to implement,
229 where document updates are validated to check if the user is listed in
230 an “author” field in the existing document. More dynamic models are
231 also possible, like checking a separate user account profile for per‐
232 mission settings.
233 The update validations are enforced for both live usage and replicated
235 updates, ensuring security and data validation in a shared, distributed
236 system.
237 SEE ALSO:
239 vdufun
240 Distributed Updates and Replication
242 CouchDB is a peer-based distributed database system. It allows users
243 and servers to access and update the same shared data while discon‐
244 nected. Those changes can then be replicated bi-directionally later.
245 The CouchDB document storage, view and security models are designed to
247 work together to make true bi-directional replication efficient and
248 reliable. Both documents and designs can replicate, allowing full
249 database applications (including application design, logic and data) to
250 be replicated to laptops for offline use, or replicated to servers in
251 remote offices where slow or unreliable connections make sharing data
252 difficult.
253 The replication process is incremental. At the database level, replica‐
255 tion only examines documents updated since the last replication. If
256 replication fails at any step, due to network problems or crash for
257 example, the next replication restarts at the last checkpoint.
258 Partial replicas can be created and maintained. Replication can be fil‐
260 tered by a JavaScript function, so that only particular documents or
261 those meeting specific criteria are replicated. This can allow users to
262 take subsets of a large shared database application offline for their
263 own use, while maintaining normal interaction with the application and
264 that subset of data.
265 Conflicts
267 Conflict detection and management are key issues for any distributed
268 edit system. The CouchDB storage system treats edit conflicts as a com‐
269 mon state, not an exceptional one. The conflict handling model is sim‐
270 ple and “non-destructive” while preserving single document semantics
271 and allowing for decentralized conflict resolution.
272 CouchDB allows for any number of conflicting documents to exist simul‐
274 taneously in the database, with each database instance deterministi‐
275 cally deciding which document is the “winner” and which are conflicts.
276 Only the winning document can appear in views, while “losing” conflicts
277 are still accessible and remain in the database until deleted or purged
278 during database compaction. Because conflict documents are still regu‐
279 lar documents, they replicate just like regular documents and are sub‐
280 ject to the same security and validation rules.
281 When distributed edit conflicts occur, every database replica sees the
283 same winning revision and each has the opportunity to resolve the con‐
284 flict. Resolving conflicts can be done manually or, depending on the
285 nature of the data and the conflict, by automated agents. The system
286 makes decentralized conflict resolution possible while maintaining sin‐
287 gle document database semantics.
288 Conflict management continues to work even if multiple disconnected
290 users or agents attempt to resolve the same conflicts. If resolved con‐
291 flicts result in more conflicts, the system accommodates them in the
292 same manner, determining the same winner on each machine and maintain‐
293 ing single document semantics.
294 SEE ALSO:
296 replication/conflicts
297 Applications
299 Using just the basic replication model, many traditionally single
300 server database applications can be made distributed with almost no
301 extra work. CouchDB replication is designed to be immediately useful
302 for basic database applications, while also being extendable for more
303 elaborate and full-featured uses.
304 With very little database work, it is possible to build a distributed
306 document management application with granular security and full revi‐
307 sion histories. Updates to documents can be implemented to exploit
308 incremental field and blob replication, where replicated updates are
309 nearly as efficient and incremental as the actual edit differences
310 (“diffs”).
311 Implementation
313 CouchDB is built on the Erlang OTP platform, a functional, concurrent
314 programming language and development platform. Erlang was developed for
315 real-time telecom applications with an extreme emphasis on reliability
316 and availability.
317 Both in syntax and semantics, Erlang is very different from conven‐
319 tional programming languages like C or Java. Erlang uses lightweight
320 “processes” and message passing for concurrency, it has no shared state
321 threading and all data is immutable. The robust, concurrent nature of
322 Erlang is ideal for a database server.
323 CouchDB is designed for lock-free concurrency, in the conceptual model
325 and the actual Erlang implementation. Reducing bottlenecks and avoiding
326 locks keeps the entire system working predictably under heavy loads.
327 CouchDB can accommodate many clients replicating changes, opening and
328 updating documents, and querying views whose indexes are simultaneously
329 being refreshed for other clients, without needing locks.
330 For higher availability and more concurrent users, CouchDB is designed
332 for “shared nothing” clustering. In a “shared nothing” cluster, each
333 machine is independent and replicates data with its cluster mates,
334 allowing individual server failures with zero downtime. And because
335 consistency scans and fix-ups aren’t needed on restart, if the entire
336 cluster fails – due to a power outage in a datacenter, for example –
337 the entire CouchDB distributed system becomes immediately available
338 after a restart.
339 CouchDB is built from the start with a consistent vision of a distrib‐
341 uted document database system. Unlike cumbersome attempts to bolt dis‐
342 tributed features on top of the same legacy models and databases, it is
343 the result of careful ground-up design, engineering and integration.
344 The document, view, security and replication models, the special pur‐
345 pose query language, the efficient and robust disk layout and the con‐
346 current and reliable nature of the Erlang platform are all carefully
347 integrated for a reliable and efficient system.
348 Why CouchDB?
350 Apache CouchDB is one of a new breed of database management systems.
351 This topic explains why there’s a need for new systems as well as the
352 motivations behind building CouchDB.
353 As CouchDB developers, we’re naturally very excited to be using
355 CouchDB. In this topic we’ll share with you the reasons for our enthu‐
356 siasm. We’ll show you how CouchDB’s schema-free document model is a
357 better fit for common applications, how the built-in query engine is a
358 powerful way to use and process your data, and how CouchDB’s design
359 lends itself to modularization and scalability.
360 Relax
362 If there’s one word to describe CouchDB, it is relax. It is the byline
363 to CouchDB’s official logo and when you start CouchDB, you see:
364 Apache CouchDB has started. Time to relax.
366 Why is relaxation important? Developer productivity roughly doubled in
368 the last five years. The chief reason for the boost is more powerful
369 tools that are easier to use. Take Ruby on Rails as an example. It is
370 an infinitely complex framework, but it’s easy to get started with.
371 Rails is a success story because of the core design focus on ease of
372 use. This is one reason why CouchDB is relaxing: learning CouchDB and
373 understanding its core concepts should feel natural to most everybody
374 who has been doing any work on the Web. And it is still pretty easy to
375 explain to non-technical people.
376 Getting out of the way when creative people try to build specialized
378 solutions is in itself a core feature and one thing that CouchDB aims
379 to get right. We found existing tools too cumbersome to work with dur‐
380 ing development or in production, and decided to focus on making
381 CouchDB easy, even a pleasure, to use.
382 Another area of relaxation for CouchDB users is the production setting.
384 If you have a live running application, CouchDB again goes out of its
385 way to avoid troubling you. Its internal architecture is fault-toler‐
386 ant, and failures occur in a controlled environment and are dealt with
387 gracefully. Single problems do not cascade through an entire server
388 system but stay isolated in single requests.
389 CouchDB’s core concepts are simple (yet powerful) and well understood.
391 Operations teams (if you have a team; otherwise, that’s you) do not
392 have to fear random behavior and untraceable errors. If anything should
393 go wrong, you can easily find out what the problem is, but these situa‐
394 tions are rare.
395 CouchDB is also designed to handle varying traffic gracefully. For
397 instance, if a website is experiencing a sudden spike in traffic,
398 CouchDB will generally absorb a lot of concurrent requests without
399 falling over. It may take a little more time for each request, but they
400 all get answered. When the spike is over, CouchDB will work with regu‐
401 lar speed again.
402 The third area of relaxation is growing and shrinking the underlying
404 hardware of your application. This is commonly referred to as scaling.
405 CouchDB enforces a set of limits on the programmer. On first look,
406 CouchDB might seem inflexible, but some features are left out by design
407 for the simple reason that if CouchDB supported them, it would allow a
408 programmer to create applications that couldn’t deal with scaling up or
409 down.
410 NOTE:
412 CouchDB doesn’t let you do things that would get you in trouble
413 later on. This sometimes means you’ll have to unlearn best prac‐
414 tices you might have picked up in your current or past work.
415 A Different Way to Model Your Data
417 We believe that CouchDB will drastically change the way you build docu‐
418 ment-based applications. CouchDB combines an intuitive document storage
419 model with a powerful query engine in a way that’s so simple you’ll
420 probably be tempted to ask, “Why has no one built something like this
421 before?”
422 Django may be built for the Web, but CouchDB is built of the Web.
423 I’ve never seen software that so completely embraces the philoso‐
424 phies behind HTTP. CouchDB makes Django look old-school in the same
425 way that Django makes ASP look outdated. — Jacob Kaplan-Moss,
426 Django developer
427 CouchDB’s design borrows heavily from web architecture and the concepts
429 of resources, methods, and representations. It augments this with pow‐
430 erful ways to query, map, combine, and filter your data. Add fault tol‐
431 erance, extreme scalability, and incremental replication, and CouchDB
432 defines a sweet spot for document databases.
433 A Better Fit for Common Applications
435 We write software to improve our lives and the lives of others. Usually
436 this involves taking some mundane information such as contacts,
437 invoices, or receipts and manipulating it using a computer application.
438 CouchDB is a great fit for common applications like this because it
439 embraces the natural idea of evolving, self-contained documents as the
440 very core of its data model.
441 Self-Contained Data
443 An invoice contains all the pertinent information about a single trans‐
444 action the seller, the buyer, the date, and a list of the items or ser‐
445 vices sold. As shown in Figure 1. Self-contained documents, there’s no
446 abstract reference on this piece of paper that points to some other
447 piece of paper with the seller’s name and address. Accountants appreci‐
448 ate the simplicity of having everything in one place. And given the
449 choice, programmers appreciate that, too.
450 [image: Self-contained documents] [image] Figure 1. Self-contained
451 documents.UNINDENT
452 Yet using references is exactly how we model our data in a relational
454 database! Each invoice is stored in a table as a row that refers to
455 other rows in other tables one row for seller information, one for
456 the buyer, one row for each item billed, and more rows still to
457 describe the item details, manufacturer details, and so on and so
458 forth.
459 This isn’t meant as a detraction of the relational model, which is
461 widely applicable and extremely useful for a number of reasons. Hope‐
462 fully, though, it illustrates the point that sometimes your model may
463 not “fit” your data in the way it occurs in the real world.
464 Let’s take a look at the humble contact database to illustrate a dif‐
466 ferent way of modeling data, one that more closely “fits” its
467 real-world counterpart – a pile of business cards. Much like our
468 invoice example, a business card contains all the important informa‐
469 tion, right there on the cardstock. We call this “self-contained”
470 data, and it’s an important concept in understanding document data‐
471 bases like CouchDB.
472 Syntax and Semantics
474 Most business cards contain roughly the same information – someone’s
475 identity, an affiliation, and some contact information. While the exact
476 form of this information can vary between business cards, the general
477 information being conveyed remains the same, and we’re easily able to
478 recognize it as a business card. In this sense, we can describe a busi‐
479 ness card as a real-world document.
480 Jan’s business card might contain a phone number but no fax number,
482 whereas J. Chris’s business card contains both a phone and a fax num‐
483 ber. Jan does not have to make his lack of a fax machine explicit by
484 writing something as ridiculous as “Fax: None” on the business card.
485 Instead, simply omitting a fax number implies that he doesn’t have one.
486 We can see that real-world documents of the same type, such as business
488 cards, tend to be very similar in semantics – the sort of information
489 they carry, but can vary hugely in syntax, or how that information is
490 structured. As human beings, we’re naturally comfortable dealing with
491 this kind of variation.
492 While a traditional relational database requires you to model your data
494 up front, CouchDB’s schema-free design unburdens you with a powerful
495 way to aggregate your data after the fact, just like we do with
496 real-world documents. We’ll look in depth at how to design applications
497 with this underlying storage paradigm.
498 Building Blocks for Larger Systems
500 CouchDB is a storage system useful on its own. You can build many
501 applications with the tools CouchDB gives you. But CouchDB is designed
502 with a bigger picture in mind. Its components can be used as building
503 blocks that solve storage problems in slightly different ways for
504 larger and more complex systems.
505 Whether you need a system that’s crazy fast but isn’t too concerned
507 with reliability (think logging), or one that guarantees storage in two
508 or more physically separated locations for reliability, but you’re
509 willing to take a performance hit, CouchDB lets you build these sys‐
510 tems.
511 There are a multitude of knobs you could turn to make a system work
513 better in one area, but you’ll affect another area when doing so. One
514 example would be the CAP theorem discussed in intro/consistency. To
515 give you an idea of other things that affect storage systems, see
516 Figure 2 and Figure 3.
517 By reducing latency for a given system (and that is true not only for
519 storage systems), you affect concurrency and throughput capabilities.
520 [image: Throughput, latency, or concurrency] [image] Figure 2.
521 Throughput, latency, or concurrency.UNINDENT
522 [image: Scaling: read requests, write requests, or data] [image] Fig‐
523 ure 3. Scaling: read requests, write requests, or data.UNINDENT
524 When you want to scale out, there are three distinct issues to deal
526 with: scaling read requests, write requests, and data. Orthogonal to
527 all three and to the items shown in Figure 2 and Figure 3 are many
528 more attributes like reliability or simplicity. You can draw many of
529 these graphs that show how different features or attributes pull into
530 different directions and thus shape the system they describe.
531 CouchDB is very flexible and gives you enough building blocks to cre‐
533 ate a system shaped to suit your exact problem. That’s not saying
534 that CouchDB can be bent to solve any problem – CouchDB is no silver
535 bullet – but in the area of data storage, it can get you a long way.
536 CouchDB Replication
538 CouchDB replication is one of these building blocks. Its fundamental
539 function is to synchronize two or more CouchDB databases. This may
540 sound simple, but the simplicity is key to allowing replication to
541 solve a number of problems: reliably synchronize databases between mul‐
542 tiple machines for redundant data storage; distribute data to a cluster
543 of CouchDB instances that share a subset of the total number of
544 requests that hit the cluster (load balancing); and distribute data
545 between physically distant locations, such as one office in New York
546 and another in Tokyo.
547 CouchDB replication uses the same REST API all clients use. HTTP is
549 ubiquitous and well understood. Replication works incrementally; that
550 is, if during replication anything goes wrong, like dropping your net‐
551 work connection, it will pick up where it left off the next time it
552 runs. It also only transfers data that is needed to synchronize data‐
553 bases.
554 A core assumption CouchDB makes is that things can go wrong, like net‐
556 work connection troubles, and it is designed for graceful error recov‐
557 ery instead of assuming all will be well. The replication system’s
558 incremental design shows that best. The ideas behind “things that can
559 go wrong” are embodied in the Fallacies of Distributed Computing:
560 · The network is reliable.
562 · Latency is zero.
564 · Bandwidth is infinite.
566 · The network is secure.
568 · Topology doesn’t change.
570 · There is one administrator.
572 · Transport cost is zero.
574 · The network is homogeneous.
576 Existing tools often try to hide the fact that there is a network and
578 that any or all of the previous conditions don’t exist for a particular
579 system. This usually results in fatal error scenarios when something
580 finally goes wrong. In contrast, CouchDB doesn’t try to hide the net‐
581 work; it just handles errors gracefully and lets you know when actions
582 on your end are required.
583 Local Data Is King
585 CouchDB takes quite a few lessons learned from the Web, but there is
586 one thing that could be improved about the Web: latency. Whenever you
587 have to wait for an application to respond or a website to render, you
588 almost always wait for a network connection that isn’t as fast as you
589 want it at that point. Waiting a few seconds instead of milliseconds
590 greatly affects user experience and thus user satisfaction.
591 What do you do when you are offline? This happens all the time – your
593 DSL or cable provider has issues, or your iPhone, G1, or Blackberry has
594 no bars, and no connectivity means no way to get to your data.
595 CouchDB can solve this scenario as well, and this is where scaling is
597 important again. This time it is scaling down. Imagine CouchDB
598 installed on phones and other mobile devices that can synchronize data
599 with centrally hosted CouchDBs when they are on a network. The synchro‐
600 nization is not bound by user interface constraints like sub-second
601 response times. It is easier to tune for high bandwidth and higher
602 latency than for low bandwidth and very low latency. Mobile applica‐
603 tions can then use the local CouchDB to fetch data, and since no remote
604 networking is required for that, latency is low by default.
605 Can you really use CouchDB on a phone? Erlang, CouchDB’s implementation
607 language has been designed to run on embedded devices magnitudes
608 smaller and less powerful than today’s phones.
609 Wrapping Up
611 The next document intro/consistency further explores the distributed
612 nature of CouchDB. We should have given you enough bites to whet your
613 interest. Let’s go!
614 Eventual Consistency
616 In the previous document intro/why, we saw that CouchDB’s flexibility
617 allows us to evolve our data as our applications grow and change. In
618 this topic, we’ll explore how working “with the grain” of CouchDB pro‐
619 motes simplicity in our applications and helps us naturally build scal‐
620 able, distributed systems.
621 Working with the Grain
623 A distributed system is a system that operates robustly over a wide
624 network. A particular feature of network computing is that network
625 links can potentially disappear, and there are plenty of strategies for
626 managing this type of network segmentation. CouchDB differs from others
627 by accepting eventual consistency, as opposed to putting absolute con‐
628 sistency ahead of raw availability, like RDBMS or Paxos. What these
629 systems have in common is an awareness that data acts differently when
630 many people are accessing it simultaneously. Their approaches differ
631 when it comes to which aspects of consistency, availability, or parti‐
632 tion tolerance they prioritize.
633 Engineering distributed systems is tricky. Many of the caveats and
635 “gotchas” you will face over time aren’t immediately obvious. We don’t
636 have all the solutions, and CouchDB isn’t a panacea, but when you work
637 with CouchDB’s grain rather than against it, the path of least resis‐
638 tance leads you to naturally scalable applications.
639 Of course, building a distributed system is only the beginning. A web‐
641 site with a database that is available only half the time is next to
642 worthless. Unfortunately, the traditional relational database approach
643 to consistency makes it very easy for application programmers to rely
644 on global state, global clocks, and other high availability no-nos,
645 without even realizing that they’re doing so. Before examining how
646 CouchDB promotes scalability, we’ll look at the constraints faced by a
647 distributed system. After we’ve seen the problems that arise when parts
648 of your application can’t rely on being in constant contact with each
649 other, we’ll see that CouchDB provides an intuitive and useful way for
650 modeling applications around high availability.
651 The CAP Theorem
653 The CAP theorem describes a few different strategies for distributing
654 application logic across networks. CouchDB’s solution uses replication
655 to propagate application changes across participating nodes. This is a
656 fundamentally different approach from consensus algorithms and rela‐
657 tional databases, which operate at different intersections of consis‐
658 tency, availability, and partition tolerance.
659 The CAP theorem, shown in Figure 1. The CAP theorem, identifies three
661 distinct concerns:
662 · Consistency: All database clients see the same data, even with con‐
664 current updates.
665 · Availability: All database clients are able to access some version of
667 the data.
668 · Partition tolerance: The database can be split over multiple servers.
670 Pick two.
672 [image: The CAP theorem] [image] Figure 1. The CAP theorem.UNINDENT
673 When a system grows large enough that a single database node is
675 unable to handle the load placed on it, a sensible solution is to add
676 more servers. When we add nodes, we have to start thinking about how
677 to partition data between them. Do we have a few databases that share
678 exactly the same data? Do we put different sets of data on different
679 database servers? Do we let only certain database servers write data
680 and let others handle the reads?
681 Regardless of which approach we take, the one problem we’ll keep
683 bumping into is that of keeping all these database servers in sync.
684 If you write some information to one node, how are you going to make
685 sure that a read request to another database server reflects this
686 newest information? These events might be milliseconds apart. Even
687 with a modest collection of database servers, this problem can become
688 extremely complex.
689 When it’s absolutely critical that all clients see a consistent view
691 of the database, the users of one node will have to wait for any
692 other nodes to come into agreement before being able to read or write
693 to the database. In this instance, we see that availability takes a
694 backseat to consistency. However, there are situations where avail‐
695 ability trumps consistency:
696 Each node in a system should be able to make decisions purely based
697 on local state. If you need to do something under high load with
698 failures occurring and you need to reach agreement, you’re lost. If
699 you’re concerned about scalability, any algorithm that forces you to
700 run agreement will eventually become your bottleneck. Take that as a
701 given. — Werner Vogels, Amazon CTO and Vice President
702 If availability is a priority, we can let clients write data to one
704 node of the database without waiting for other nodes to come into
705 agreement. If the database knows how to take care of reconciling these
706 operations between nodes, we achieve a sort of “eventual consistency”
707 in exchange for high availability. This is a surprisingly applicable
708 trade-off for many applications.
709 Unlike traditional relational databases, where each action performed is
711 necessarily subject to database-wide consistency checks, CouchDB makes
712 it really simple to build applications that sacrifice immediate consis‐
713 tency for the huge performance improvements that come with simple dis‐
714 tribution.
715 Local Consistency
717 Before we attempt to understand how CouchDB operates in a cluster, it’s
718 important that we understand the inner workings of a single CouchDB
719 node. The CouchDB API is designed to provide a convenient but thin
720 wrapper around the database core. By taking a closer look at the struc‐
721 ture of the database core, we’ll have a better understanding of the API
722 that surrounds it.
723 The Key to Your Data
725 At the heart of CouchDB is a powerful B-tree storage engine. A B-tree
726 is a sorted data structure that allows for searches, insertions, and
727 deletions in logarithmic time. As Figure 2. Anatomy of a view request
728 illustrates, CouchDB uses this B-tree storage engine for all internal
729 data, documents, and views. If we understand one, we will understand
730 them all.
731 [image: Anatomy of a view request] [image] Figure 2. Anatomy of a
732 view request.UNINDENT
733 CouchDB uses MapReduce to compute the results of a view. MapReduce
735 makes use of two functions, “map” and “reduce”, which are applied to
736 each document in isolation. Being able to isolate these operations
737 means that view computation lends itself to parallel and incremental
738 computation. More important, because these functions produce
739 key/value pairs, CouchDB is able to insert them into the B-tree stor‐
740 age engine, sorted by key. Lookups by key, or key range, are
741 extremely efficient operations with a B-tree, described in big O
742 notation as O(log N) and O(log N + K), respectively.
743 In CouchDB, we access documents and view results by key or key range.
745 This is a direct mapping to the underlying operations performed on
746 CouchDB’s B-tree storage engine. Along with document inserts and
747 updates, this direct mapping is the reason we describe CouchDB’s API
748 as being a thin wrapper around the database core.
749 Being able to access results by key alone is a very important
751 restriction because it allows us to make huge performance gains. As
752 well as the massive speed improvements, we can partition our data
753 over multiple nodes, without affecting our ability to query each node
754 in isolation. BigTable, Hadoop, SimpleDB, and memcached restrict
755 object lookups by key for exactly these reasons.
756 No Locking
758 A table in a relational database is a single data structure. If you
759 want to modify a table – say, update a row – the database system must
760 ensure that nobody else is trying to update that row and that nobody
761 can read from that row while it is being updated. The common way to
762 handle this uses what’s known as a lock. If multiple clients want to
763 access a table, the first client gets the lock, making everybody else
764 wait. When the first client’s request is processed, the next client is
765 given access while everybody else waits, and so on. This serial execu‐
766 tion of requests, even when they arrived in parallel, wastes a signifi‐
767 cant amount of your server’s processing power. Under high load, a
768 relational database can spend more time figuring out who is allowed to
769 do what, and in which order, than it does doing any actual work.
770 NOTE:
772 Modern relational databases avoid locks by implementing MVCC under
773 the hood, but hide it from the end user, requiring them to coordi‐
774 nate concurrent changes of single rows or fields.
775 Instead of locks, CouchDB uses Multi-Version Concurrency Control (MVCC)
777 to manage concurrent access to the database. Figure 3. MVCC means no
778 locking illustrates the differences between MVCC and traditional lock‐
779 ing mechanisms. MVCC means that CouchDB can run at full speed, all the
780 time, even under high load. Requests are run in parallel, making excel‐
781 lent use of every last drop of processing power your server has to
782 offer.
783 [image: MVCC means no locking] [image] Figure 3. MVCC means no lock‐
784 ing.UNINDENT
785 Documents in CouchDB are versioned, much like they would be in a reg‐
787 ular version control system such as Subversion. If you want to change
788 a value in a document, you create an entire new version of that docu‐
789 ment and save it over the old one. After doing this, you end up with
790 two versions of the same document, one old and one new.
791 How does this offer an improvement over locks? Consider a set of
793 requests wanting to access a document. The first request reads the
794 document. While this is being processed, a second request changes
795 the document. Since the second request includes a completely new
796 version of the document, CouchDB can simply append it to the database
797 without having to wait for the read request to finish.
798 When a third request wants to read the same document, CouchDB will
800 point it to the new version that has just been written. During this
801 whole process, the first request could still be reading the original
802 version.
803 A read request will always see the most recent snapshot of your data‐
805 base at the time of the beginning of the request.
806 Validation
808 As application developers, we have to think about what sort of input we
809 should accept and what we should reject. The expressive power to do
810 this type of validation over complex data within a traditional rela‐
811 tional database leaves a lot to be desired. Fortunately, CouchDB pro‐
812 vides a powerful way to perform per-document validation from within the
813 database.
814 CouchDB can validate documents using JavaScript functions similar to
816 those used for MapReduce. Each time you try to modify a document,
817 CouchDB will pass the validation function a copy of the existing docu‐
818 ment, a copy of the new document, and a collection of additional infor‐
819 mation, such as user authentication details. The validation function
820 now has the opportunity to approve or deny the update.
821 By working with the grain and letting CouchDB do this for us, we save
823 ourselves a tremendous amount of CPU cycles that would otherwise have
824 been spent serializing object graphs from SQL, converting them into
825 domain objects, and using those objects to do application-level valida‐
826 tion.
827 Distributed Consistency
829 Maintaining consistency within a single database node is relatively
830 easy for most databases. The real problems start to surface when you
831 try to maintain consistency between multiple database servers. If a
832 client makes a write operation on server A, how do we make sure that
833 this is consistent with server B, or C, or D? For relational databases,
834 this is a very complex problem with entire books devoted to its solu‐
835 tion. You could use multi-master, single-master, partitioning, shard‐
836 ing, write-through caches, and all sorts of other complex techniques.
837 Incremental Replication
839 CouchDB’s operations take place within the context of a single docu‐
840 ment. As CouchDB achieves eventual consistency between multiple data‐
841 bases by using incremental replication you no longer have to worry
842 about your database servers being able to stay in constant communica‐
843 tion. Incremental replication is a process where document changes are
844 periodically copied between servers. We are able to build what’s known
845 as a shared nothing cluster of databases where each node is independent
846 and self-sufficient, leaving no single point of contention across the
847 system.
848 Need to scale out your CouchDB database cluster? Just throw in another
850 server.
851 As illustrated in Figure 4. Incremental replication between CouchDB
853 nodes, with CouchDB’s incremental replication, you can synchronize your
854 data between any two databases however you like and whenever you like.
855 After replication, each database is able to work independently.
856 You could use this feature to synchronize database servers within a
858 cluster or between data centers using a job scheduler such as cron, or
859 you could use it to synchronize data with your laptop for offline work
860 as you travel. Each database can be used in the usual fashion, and
861 changes between databases can be synchronized later in both directions.
862 [image: Incremental replication between CouchDB nodes] [image] Figure
863 4. Incremental replication between CouchDB nodes.UNINDENT
864 What happens when you change the same document in two different data‐
866 bases and want to synchronize these with each other? CouchDB’s repli‐
867 cation system comes with automatic conflict detection and resolution.
868 When CouchDB detects that a document has been changed in both data‐
869 bases, it flags this document as being in conflict, much like they
870 would be in a regular version control system.
871 This isn’t as troublesome as it might first sound. When two versions
873 of a document conflict during replication, the winning version is
874 saved as the most recent version in the document’s history. Instead
875 of throwing the losing version away, as you might expect, CouchDB
876 saves this as a previous version in the document’s history, so that
877 you can access it if you need to. This happens automatically and con‐
878 sistently, so both databases will make exactly the same choice.
879 It is up to you to handle conflicts in a way that makes sense for
881 your application. You can leave the chosen document versions in
882 place, revert to the older version, or try to merge the two versions
883 and save the result.
884 Case Study
886 Greg Borenstein, a friend and coworker, built a small library for con‐
887 verting Songbird playlists to JSON objects and decided to store these
888 in CouchDB as part of a backup application. The completed software uses
889 CouchDB’s MVCC and document revisions to ensure that Songbird playlists
890 are backed up robustly between nodes.
891 NOTE:
893 Songbird is a free software media player with an integrated web
894 browser, based on the Mozilla XULRunner platform. Songbird is avail‐
895 able for Microsoft Windows, Apple Mac OS X, Solaris, and Linux.
896 Let’s examine the workflow of the Songbird backup application, first as
898 a user backing up from a single computer, and then using Songbird to
899 synchronize playlists between multiple computers. We’ll see how docu‐
900 ment revisions turn what could have been a hairy problem into something
901 that just works.
902 The first time we use this backup application, we feed our playlists to
904 the application and initiate a backup. Each playlist is converted to a
905 JSON object and handed to a CouchDB database. As illustrated in Figure
906 5. Backing up to a single database, CouchDB hands back the document ID
907 and revision of each playlist as it’s saved to the database.
908 [image: Backing up to a single database] [image] Figure 5. Backing up
909 to a single database.UNINDENT
910 After a few days, we find that our playlists have been updated and we
912 want to back up our changes. After we have fed our playlists to the
913 backup application, it fetches the latest versions from CouchDB,
914 along with the corresponding document revisions. When the application
915 hands back the new playlist document, CouchDB requires that the docu‐
916 ment revision is included in the request.
917 CouchDB then makes sure that the document revision handed to it in
919 the request matches the current revision held in the database.
920 Because CouchDB updates the revision with every modification, if
921 these two are out of sync it suggests that someone else has made
922 changes to the document between the time we requested it from the
923 database and the time we sent our updates. Making changes to a docu‐
924 ment after someone else has modified it without first inspecting
925 those changes is usually a bad idea.
926 Forcing clients to hand back the correct document revision is the
928 heart of CouchDB’s optimistic concurrency.
929 We have a laptop we want to keep synchronized with our desktop com‐
931 puter. With all our playlists on our desktop, the first step is to
932 “restore from backup” onto our laptop. This is the first time we’ve
933 done this, so afterward our laptop should hold an exact replica of
934 our desktop playlist collection.
935 After editing our Argentine Tango playlist on our laptop to add a few
937 new songs we’ve purchased, we want to save our changes. The backup
938 application replaces the playlist document in our laptop CouchDB
939 database and a new document revision is generated. A few days later,
940 we remember our new songs and want to copy the playlist across to our
941 desktop computer. As illustrated in Figure 6. Synchronizing between
942 two databases, the backup application copies the new document and the
943 new revision to the desktop CouchDB database. Both CouchDB databases
944 now have the same document revision.
945 [image: Synchronizing between two databases] [image] Figure 6. Syn‐
946 chronizing between two databases.UNINDENT
947 Because CouchDB tracks document revisions, it ensures that updates
949 like these will work only if they are based on current information.
950 If we had made modifications to the playlist backups between synchro‐
951 nization, things wouldn’t go as smoothly.
952 We back up some changes on our laptop and forget to synchronize. A
954 few days later, we’re editing playlists on our desktop computer, make
955 a backup, and want to synchronize this to our laptop. As illustrated
956 in Figure 7. Synchronization conflicts between two databases, when
957 our backup application tries to replicate between the two databases,
958 CouchDB sees that the changes being sent from our desktop computer
959 are modifications of out-of-date documents and helpfully informs us
960 that there has been a conflict.
961 Recovering from this error is easy to accomplish from an application
963 perspective. Just download CouchDB’s version of the playlist and pro‐
964 vide an opportunity to merge the changes or save local modifications
965 into a new playlist.
966 [image: Synchronization conflicts between two databases] [image] Fig‐
967 ure 7. Synchronization conflicts between two databases.UNINDENT
968 Wrapping Up
970 CouchDB’s design borrows heavily from web architecture and the lessons
971 learned deploying massively distributed systems on that architecture.
972 By understanding why this architecture works the way it does, and by
973 learning to spot which parts of your application can be easily distrib‐
974 uted and which parts cannot, you’ll enhance your ability to design dis‐
975 tributed and scalable applications, with CouchDB or without it.
976 We’ve covered the main issues surrounding CouchDB’s consistency model
978 and hinted at some of the benefits to be had when you work with CouchDB
979 and not against it. But enough theory – let’s get up and running and
980 see what all the fuss is about!
981 cURL: Your Command Line Friend
983 The curl utility is a command line tool available on Unix, Linux, Mac
984 OS X, Windows, and many other platforms. curl provides easy access to
985 the HTTP protocol (among others) directly from the command line and is
986 therefore an ideal way of interacting with CouchDB over the HTTP REST
987 API.
988 For simple GET requests you can supply the URL of the request. For
990 example, to get the database information:
991 shell> curl http://admin:password@127.0.0.1:5984
993 This returns the database information (formatted in the output below
995 for clarity):
996 {
998 "couchdb": "Welcome",
999 "version": "3.0.0",
1000 "git_sha": "83bdcf693",
1001 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
1002 "features": [
1003 "access-ready",
1004 "partitioned",
1005 "pluggable-storage-engines",
1006 "reshard",
1007 "scheduler"
1008 ],
1009 "vendor": {
1010 "name": "The Apache Software Foundation"
1011 }
1012 }
1013 NOTE:
1015 For some URLs, especially those that include special characters such
1016 as ampersand, exclamation mark, or question mark, you should quote
1017 the URL you are specifying on the command line. For example:
1018 shell> curl 'http://couchdb:5984/_uuids?count=5'
1020 NOTE:
1022 On Microsoft Windows, use double-quotes anywhere you see sin‐
1023 gle-quotes in the following examples. Use doubled double-quotes (“”)
1024 anywhere you see single quotes. For example, if you see:
1025 shell> curl -X PUT 'http:/127.0.0.1:5984/demo/doc' -d '{"motto": "I love gnomes"}'
1027 you should replace it with:
1029 shell> curl -X PUT "http://127.0.0.1:5984/demo/doc" -d "{""motto"": ""I love gnomes""}"
1031 If you prefer, ^" and \" may be used to escape the double-quote
1033 character in quoted strings instead.
1034 You can explicitly set the HTTP command using the -X command line
1036 option. For example, when creating a database, you set the name of the
1037 database in the URL you send using a PUT request:
1038 shell> curl -X PUT http://user:pass@127.0.0.1:5984/demo
1040 {"ok":true}
1041 But to obtain the database information you use a GET request (with the
1043 return information formatted for clarity):
1044 shell> curl -X GET http://user:pass@127.0.0.1:5984/demo
1046 {
1047 "compact_running" : false,
1048 "doc_count" : 0,
1049 "db_name" : "demo",
1050 "purge_seq" : 0,
1051 "committed_update_seq" : 0,
1052 "doc_del_count" : 0,
1053 "disk_format_version" : 5,
1054 "update_seq" : 0,
1055 "instance_start_time" : "0",
1056 "disk_size" : 79
1057 }
1058 For certain operations, you must specify the content type of request,
1060 which you do by specifying the Content-Type header using the -H com‐
1061 mand-line option:
1062 shell> curl -H 'Content-Type: application/json' http://127.0.0.1:5984/_uuids
1064 You can also submit ‘payload’ data, that is, data in the body of the
1066 HTTP request using the -d option. This is useful if you need to submit
1067 JSON structures, for example document data, as part of the request. For
1068 example, to submit a simple document to the demo database:
1069 shell> curl -H 'Content-Type: application/json' \
1071 -X POST http://user:pass@127.0.0.1:5984/demo \
1072 -d '{"company": "Example, Inc."}'
1073 {"ok":true,"id":"8843faaf0b831d364278331bc3001bd8",
1074 "rev":"1-33b9fbce46930280dab37d672bbc8bb9"}
1075 In the above example, the argument after the -d option is the JSON of
1077 the document we want to submit.
1078 The document can be accessed by using the automatically generated docu‐
1080 ment ID that was returned:
1081 shell> curl -X GET http://user:pass@127.0.0.1:5984/demo/8843faaf0b831d364278331bc3001bd8
1083 {"_id":"8843faaf0b831d364278331bc3001bd8",
1084 "_rev":"1-33b9fbce46930280dab37d672bbc8bb9",
1085 "company":"Example, Inc."}
1086 The API samples in the api/basics show the HTTP command, URL and any
1088 payload information that needs to be submitted (and the expected return
1089 value). All of these examples can be reproduced using curl with the
1090 command-line examples shown above.
1091 Security
1093 In this document, we’ll look at the basic security mechanisms in
1094 CouchDB: Basic Authentication and Cookie Authentication. This is how
1095 CouchDB handles users and protects their credentials.
1096 Authentication
1098 CouchDB has the idea of an admin user (e.g. an administrator, a super
1099 user, or root) that is allowed to do anything to a CouchDB installa‐
1100 tion. By default, one admin user must be created for CouchDB to start
1101 up successfully.
1102 CouchDB also defines a set of requests that only admin users are
1104 allowed to do. If you have defined one or more specific admin users,
1105 CouchDB will ask for identification for certain requests:
1106 · Creating a database (PUT /database)
1108 · Deleting a database (DELETE /database)
1110 · Setup a database security (PUT /database/_security)
1112 · Creating a design document (PUT /database/_design/app)
1114 · Updating a design document (PUT /database/_design/app?rev=1-4E2)
1116 · Deleting a design document (DELETE /database/_design/app?rev=2-6A7)
1118 · Triggering compaction (POST /database/_compact)
1120 · Reading the task status list (GET /_active_tasks)
1122 · Restarting the server on a given node (:post:`POST
1124 /_node/{node-name}/_restart </_restart>`)
1125 · Reading the active configuration (:get:`GET /_node/{node-name}/_con‐
1127 fig </_config>`)
1128 · Updating the active configuration (:put:`PUT /_node/{node-name}/_con‐
1130 fig/section/key </_config/{section}/{key}>`)
1131 Creating a New Admin User
1133 If your installation process did not set up an admin user, you will
1134 have to add one to the configuration file by hand and restart CouchDB
1135 first. For the purposes of this example, we’ll create a default admin
1136 user with the password password.
1137 WARNING:
1139 Don’t just type in the following without thinking! Pick a good name
1140 for your administrator user that isn’t easily guessable, and pick a
1141 secure password.
1142 To the end of your etc/local.ini file, after the [admins] line, add the
1144 text admin = password, so it looks like this:
1145 [admins]
1147 admin = password
1148 (Don’t worry about the password being in plain text; we’ll come back to
1150 this.)
1151 Now, restart CouchDB using the method appropriate for your operating
1153 system. You should now be able to access CouchDB using your new admin‐
1154 istrator account:
1155 > curl http://admin:password@127.0.0.1:5984/_up
1157 {"status":"ok","seeds":{}}
1158 Great!
1160 Let’s create an admin user through the HTTP API. We’ll call her anna,
1162 and her password is secret. Note the double quotes in the following
1163 code; they are needed to denote a string value for the configuration
1164 API:
1165 > HOST="http://admin:password@127.0.0.1:5984"
1167 > NODENAME="_local"
1168 > curl -X PUT $HOST/_node/$NODENAME/_config/admins/anna -d '"secret"'
1169 ""
1170 As per the _config API’s behavior, we’re getting the previous value for
1172 the config item we just wrote. Since our admin user didn’t exist, we
1173 get an empty string.
1174 Please note that _local serves as an alias for the local node name, so
1176 for all configuration URLs, NODENAME may be set to _local, to interact
1177 with the local node’s configuration.
1178 SEE ALSO:
1180 Node Management
1181 Hashing Passwords
1183 Seeing the plain-text password is scary, isn’t it? No worries, CouchDB
1184 doesn’t show the plain-text password anywhere. It gets hashed right
1185 away. Go ahead and look at your local.ini file now. You’ll see that
1186 CouchDB has rewritten the plain text passwords so they are hashed:
1187 [admins]
1189 admin = -pbkdf2-71c01cb429088ac1a1e95f3482202622dc1e53fe,226701bece4ae0fc9a373a5e02bf5d07,10
1190 anna = -pbkdf2-2d86831c82b440b8887169bd2eebb356821d621b,5e11b9a9228414ab92541beeeacbf125,10
1191 The hash is that big, ugly, long string that starts out with -pbkdf2-.
1193 To compare a plain-text password during authentication with the stored
1195 hash, the hashing algorithm is run and the resulting hash is compared
1196 to the stored hash. The probability of two identical hashes for differ‐
1197 ent passwords is too insignificant to mention (c.f. Bruce Schneier).
1198 Should the stored hash fall into the hands of an attacker, it is, by
1199 current standards, way too inconvenient (i.e., it’d take a lot of money
1200 and time) to find the plain-text password from the hash.
1201 When CouchDB starts up, it reads a set of .ini files with config set‐
1203 tings. It loads these settings into an internal data store (not a data‐
1204 base). The config API lets you read the current configuration as well
1205 as change it and create new entries. CouchDB writes any changes back to
1206 the .ini files.
1207 The .ini files can also be edited by hand when CouchDB is not running.
1209 Instead of creating the admin user as we showed previously, you could
1210 have stopped CouchDB, opened your local.ini, added anna = secret to the
1211 admins, and restarted CouchDB. Upon reading the new line from
1212 local.ini, CouchDB would run the hashing algorithm and write back the
1213 hash to local.ini, replacing the plain-text password — just as it did
1214 for our original admin user. To make sure CouchDB only hashes
1215 plain-text passwords and not an existing hash a second time, it pre‐
1216 fixes the hash with -pbkdf2-, to distinguish between plain-text pass‐
1217 words and PBKDF2 hashed passwords. This means your plain-text password
1218 can’t start with the characters -pbkdf2-, but that’s pretty unlikely to
1219 begin with.
1220 Basic Authentication
1222 CouchDB will not allow us to create new databases unless we give the
1223 correct admin user credentials. Let’s verify:
1224 > HOST="http://127.0.0.1:5984"
1226 > curl -X PUT $HOST/somedatabase
1227 {"error":"unauthorized","reason":"You are not a server admin."}
1228 That looks about right. Now we try again with the correct credentials:
1230 > HOST="http://anna:secret@127.0.0.1:5984"
1232 > curl -X PUT $HOST/somedatabase
1233 {"ok":true}
1234 If you have ever accessed a website or FTP server that was pass‐
1236 word-protected, the username:password@ URL variant should look famil‐
1237 iar.
1238 If you are security conscious, the missing s in http:// will make you
1240 nervous. We’re sending our password to CouchDB in plain text. This is a
1241 bad thing, right? Yes, but consider our scenario: CouchDB listens on
1242 127.0.0.1 on a development box that we’re the sole user of. Who could
1243 possibly sniff our password?
1244 If you are in a production environment, however, you need to recon‐
1246 sider. Will your CouchDB instance communicate over a public network?
1247 Even a LAN shared with other collocation customers is public. There are
1248 multiple ways to secure communication between you or your application
1249 and CouchDB that exceed the scope of this documentation. CouchDB as of
1250 version 1.1.0 comes with SSL built in.
1251 SEE ALSO:
1253 Basic Authentication API Reference
1254 Cookie Authentication
1256 Basic authentication that uses plain-text passwords is nice and conve‐
1257 nient, but not very secure if no extra measures are taken. It is also a
1258 very poor user experience. If you use basic authentication to identify
1259 admins, your application’s users need to deal with an ugly, unstylable
1260 browser modal dialog that says non-professional at work more than any‐
1261 thing else.
1262 To remedy some of these concerns, CouchDB supports cookie authentica‐
1264 tion. With cookie authentication your application doesn’t have to
1265 include the ugly login dialog that the users’ browsers come with. You
1266 can use a regular HTML form to submit logins to CouchDB. Upon receipt,
1267 CouchDB will generate a one-time token that the client can use in its
1268 next request to CouchDB. When CouchDB sees the token in a subsequent
1269 request, it will authenticate the user based on the token without the
1270 need to see the password again. By default, a token is valid for 10
1271 minutes.
1272 To obtain the first token and thus authenticate a user for the first
1274 time, the username and password must be sent to the _session API. The
1275 API is smart enough to decode HTML form submissions, so you don’t have
1276 to resort to any smarts in your application.
1277 If you are not using HTML forms to log in, you need to send an HTTP
1279 request that looks as if an HTML form generated it. Luckily, this is
1280 super simple:
1281 > HOST="http://127.0.0.1:5984"
1283 > curl -vX POST $HOST/_session \
1284 -H 'Content-Type:application/x-www-form-urlencoded' \
1285 -d 'name=anna&password=secret'
1286 CouchDB replies, and we’ll give you some more detail:
1288 < HTTP/1.1 200 OK
1290 < Set-Cookie: AuthSession=YW5uYTo0QUIzOTdFQjrC4ipN-D-53hw1sJepVzcVxnriEw;
1291 < Version=1; Path=/; HttpOnly
1292 > ...
1293 <
1294 {"ok":true}
1295 A 200 OK response code tells us all is well, a Set-Cookie header
1297 includes the token we can use for the next request, and the standard
1298 JSON response tells us again that the request was successful.
1299 Now we can use this token to make another request as the same user
1301 without sending the username and password again:
1302 > curl -vX PUT $HOST/mydatabase \
1304 --cookie AuthSession=YW5uYTo0QUIzOTdFQjrC4ipN-D-53hw1sJepVzcVxnriEw \
1305 -H "X-CouchDB-WWW-Authenticate: Cookie" \
1306 -H "Content-Type:application/x-www-form-urlencoded"
1307 {"ok":true}
1308 You can keep using this token for 10 minutes by default. After 10 min‐
1310 utes you need to authenticate your user again. The token lifetime can
1311 be configured with the timeout (in seconds) setting in the
1312 couch_httpd_auth configuration section.
1313 SEE ALSO:
1315 Cookie Authentication API Reference
1316 Authentication Database
1318 You may already note that CouchDB administrators are defined within the
1319 config file and are wondering if regular users are also stored there.
1320 No, they are not. CouchDB has a special authentication database, named
1321 _users by default, that stores all registered users as JSON documents.
1322 This special database is a system database. This means that while it
1324 shares the common database API, there are some special security-related
1325 constraints applied. Below is a list of how the authentication database
1326 is different from the other databases.
1327 · Only administrators may browse list of all documents (GET
1329 /_users/_all_docs)
1330 · Only administrators may listen to changes feed (GET /_users/_changes)
1332 · Only administrators may execute design functions like views.
1334 · There is a special design document _auth that cannot be modified
1336 · Every document except the design documents represent registered
1338 CouchDB users and belong to them
1339 · Users may only access (GET /_users/org.couchdb.user:Jan) or modify
1341 (PUT /_users/org.couchdb.user:Jan) documents that they own
1342 These draconian rules are necessary since CouchDB cares about its
1344 users’ personal information and will not disclose it to just anyone.
1345 Often, user documents contain system information like login, password
1346 hash and roles, apart from sensitive personal information like real
1347 name, email, phone, special internal identifications and more. This is
1348 not information that you want to share with the World.
1349 Users Documents
1351 Each CouchDB user is stored in document format. These documents contain
1352 several mandatory fields, that CouchDB needs for authentication:
1353 · _id (string): Document ID. Contains user’s login with special prefix
1355 Why the org.couchdb.user: prefix?
1356 · derived_key (string): PBKDF2 key
1358 · name (string): User’s name aka login. Immutable e.g. you cannot
1360 rename an existing user - you have to create new one
1361 · roles (array of string): List of user roles. CouchDB doesn’t provide
1363 any built-in roles, so you’re free to define your own depending on
1364 your needs. However, you cannot set system roles like _admin there.
1365 Also, only administrators may assign roles to users - by default all
1366 users have no roles
1367 · password_sha (string): Hashed password with salt. Used for simple
1369 password_scheme
1370 · password_scheme (string): Password hashing scheme. May be simple or
1372 pbkdf2
1373 · salt (string): Hash salt. Used for simple password_scheme
1375 · type (string): Document type. Constantly has the value user
1377 Additionally, you may specify any custom fields that relate to the tar‐
1379 get user. This is a good place to store user’s private information
1380 because only the target user and CouchDB administrators may browse it.
1381 Why the org.couchdb.user: prefix?
1383 The reason there is a special prefix before a user’s login name is to
1384 have namespaces that users belong to. This prefix is designed to pre‐
1385 vent replication conflicts when you try merging two or more _user data‐
1386 bases.
1387 For current CouchDB releases, all users belong to the same
1389 org.couchdb.user namespace and this cannot be changed. This may be
1390 changed in future releases.
1391 Creating a New User
1393 Creating a new user is a very trivial operation. You just need to do a
1394 PUT request with the user’s data to CouchDB. Let’s create a user with
1395 login jan and password apple:
1396 curl -X PUT http://localhost:5984/_users/org.couchdb.user:jan \
1398 -H "Accept: application/json" \
1399 -H "Content-Type: application/json" \
1400 -d '{"name": "jan", "password": "apple", "roles": [], "type": "user"}'
1401 This curl command will produce the following HTTP request:
1403 PUT /_users/org.couchdb.user:jan HTTP/1.1
1405 Accept: application/json
1406 Content-Length: 62
1407 Content-Type: application/json
1408 Host: localhost:5984
1409 User-Agent: curl/7.31.0
1410 And CouchDB responds with:
1412 HTTP/1.1 201 Created
1414 Cache-Control: must-revalidate
1415 Content-Length: 83
1416 Content-Type: application/json
1417 Date: Fri, 27 Sep 2013 07:33:28 GMT
1418 ETag: "1-e0ebfb84005b920488fc7a8cc5470cc0"
1419 Location: http://localhost:5984/_users/org.couchdb.user:jan
1420 Server: CouchDB (Erlang OTP)
1421 {"ok":true,"id":"org.couchdb.user:jan","rev":"1-e0ebfb84005b920488fc7a8cc5470cc0"}
1423 The document was successfully created! The user jan should now exist in
1425 our database. Let’s check if this is true:
1426 curl -X POST http://localhost:5984/_session -d 'name=jan&password=apple'
1428 CouchDB should respond with:
1430 {"ok":true,"name":"jan","roles":[]}
1432 This means that the username was recognized and the password’s hash
1434 matches with the stored one. If we specify an incorrect login and/or
1435 password, CouchDB will notify us with the following error message:
1436 {"error":"unauthorized","reason":"Name or password is incorrect."}
1438 Password Changing
1440 Let’s define what is password changing from the point of view of
1441 CouchDB and the authentication database. Since “users” are “documents”,
1442 this operation is just updating the document with a special field pass‐
1443 word which contains the plain text password. Scared? No need to be. The
1444 authentication database has a special internal hook on document update
1445 which looks for this field and replaces it with the secured hash
1446 depending on the chosen password_scheme.
1447 Summarizing the above process - we need to get the document’s content,
1449 add the password field with the new password in plain text and then
1450 store the JSON result to the authentication database.
1451 curl -X GET http://localhost:5984/_users/org.couchdb.user:jan
1453 {
1455 "_id": "org.couchdb.user:jan",
1456 "_rev": "1-e0ebfb84005b920488fc7a8cc5470cc0",
1457 "derived_key": "e579375db0e0c6a6fc79cd9e36a36859f71575c3",
1458 "iterations": 10,
1459 "name": "jan",
1460 "password_scheme": "pbkdf2",
1461 "roles": [],
1462 "salt": "1112283cf988a34f124200a050d308a1",
1463 "type": "user"
1464 }
1465 Here is our user’s document. We may strip hashes from the stored docu‐
1467 ment to reduce the amount of posted data:
1468 curl -X PUT http://localhost:5984/_users/org.couchdb.user:jan \
1470 -H "Accept: application/json" \
1471 -H "Content-Type: application/json" \
1472 -H "If-Match: 1-e0ebfb84005b920488fc7a8cc5470cc0" \
1473 -d '{"name":"jan", "roles":[], "type":"user", "password":"orange"}'
1474 {"ok":true,"id":"org.couchdb.user:jan","rev":"2-ed293d3a0ae09f0c624f10538ef33c6f"}
1476 Updated! Now let’s check that the password was really changed:
1478 curl -X POST http://localhost:5984/_session -d 'name=jan&password=apple'
1480 CouchDB should respond with:
1482 {"error":"unauthorized","reason":"Name or password is incorrect."}
1484 Looks like the password apple is wrong, what about orange?
1486 curl -X POST http://localhost:5984/_session -d 'name=jan&password=orange'
1488 CouchDB should respond with:
1490 {"ok":true,"name":"jan","roles":[]}
1492 Hooray! You may wonder why this was so complex - we need to retrieve
1494 user’s document, add a special field to it, and post it back.
1495 NOTE:
1497 There is no password confirmation for API request: you should imple‐
1498 ment it in your application layer.
1499 Users Public Information
1501 New in version 1.4.
1502 Sometimes users want to share some information with the world. For
1505 instance, their contact email to let other users get in touch with
1506 them. To solve this problem, but still keep sensitive and private
1507 information secured, there is a special configuration option pub‐
1508 lic_fields. In this option you may define a comma-separated list of
1509 users document fields that will be publicly available.
1510 Normally, if you request a user document and you’re not an administra‐
1512 tor or the document’s owner, CouchDB will respond with 404 Not Found:
1513 curl http://localhost:5984/_users/org.couchdb.user:robert
1515 {"error":"not_found","reason":"missing"}
1517 This response is constant for both cases when user exists or doesn’t
1519 exist for security reasons.
1520 Now let’s share the field name. First, set up the public_fields config‐
1522 uration option. Remember, that this action requires administrator priv‐
1523 ileges. The next command will prompt you for user admin’s password:
1524 curl -X PUT http://localhost:5984/_node/nonode@nohost/_config/couch_httpd_auth/public_fields \
1526 -H "Content-Type: application/json" \
1527 -d '"name"' \
1528 -u admin
1529 What has changed? Let’s check Robert’s document once again:
1531 curl http://localhost:5984/_users/org.couchdb.user:robert
1533 {"_id":"org.couchdb.user:robert","_rev":"6-869e2d3cbd8b081f9419f190438ecbe7","name":"robert"}
1535 Good news! Now we may read the field name in every user document with‐
1537 out needing to be an administrator. Keep in mind, though, not to pub‐
1538 lish sensitive information, especially without user’s consent!
1539 Authorization
1541 Now that you have a few users who can log in, you probably want to set
1542 up some restrictions on what actions they can perform based on their
1543 identity and their roles. Each database on a CouchDB server can con‐
1544 tain its own set of authorization rules that specify which users are
1545 allowed to read and write documents, create design documents, and
1546 change certain database configuration parameters. The authorization
1547 rules are set up by a server admin and can be modified at any time.
1548 Database authorization rules assign a user into one of two classes:
1550 · members, who are allowed to read all documents and create and modify
1552 any document except for design documents.
1553 · admins, who can read and write all types of documents, modify which
1555 users are members or admins, and set certain per-database configura‐
1556 tion options.
1557 Note that a database admin is not the same as a server admin – the
1559 actions of a database admin are restricted to a specific database.
1560 When a database is first created, there are no members or admins. HTTP
1562 requests that have no authentication credentials or have credentials
1563 for a normal user are treated as members, and those with server admin
1564 credentials are treated as database admins. To change the default per‐
1565 missions, you must create a _security document in the database:
1566 > curl -X PUT http://localhost:5984/mydatabase/_security \
1568 -u anna:secret \
1569 -H "Content-Type: application/json" \
1570 -d '{"admins": { "names": [], "roles": [] }, "members": { "names": ["jan"], "roles": [] } }'
1571 The HTTP request to create the _security document must contain the cre‐
1573 dentials of a server admin. CouchDB will respond with:
1574 {"ok":true}
1576 The database is now secured against anonymous reads and writes:
1578 > curl http://localhost:5984/mydatabase/
1580 {"error":"unauthorized","reason":"You are not authorized to access this db."}
1582 You declared user “jan” as a member in this database, so he is able to
1584 read and write normal documents:
1585 > curl -u jan:apple http://localhost:5984/mydatabase/
1587 {"db_name":"mydatabase","doc_count":1,"doc_del_count":0,"update_seq":3,"purge_seq":0,
1589 "compact_running":false,"sizes":{"active":272,"disk":12376,"external":350},
1590 "instance_start_time":"0","disk_format_version":6,"committed_update_seq":3}
1591 If Jan attempted to create a design doc, however, CouchDB would return
1593 a 401 Unauthorized error because the username “jan” is not in the list
1594 of admin names and the /_users/org.couchdb.user:jan document doesn’t
1595 contain a role that matches any of the declared admin roles. If you
1596 want to promote Jan to an admin, you can update the security document
1597 to add “jan” to the names array under admin. Keeping track of individ‐
1598 ual database admin usernames is tedious, though, so you would likely
1599 prefer to create a database admin role and assign that role to the
1600 org.couchdb.user:jan user document:
1601 > curl -X PUT http://localhost:5984/mydatabase/_security \
1603 -u anna:secret \
1604 -H "Content-Type: application/json" \
1605 -d '{"admins": { "names": [], "roles": ["mydatabase_admin"] }, "members": { "names": [], "roles": [] } }'
1606 See the _security document reference page for additional details about
1608 specifying database members and admins.
1609 Getting Started
1611 In this document, we’ll take a quick tour of CouchDB’s features. We’ll
1612 create our first document and experiment with CouchDB views.
1613 All Systems Are Go!
1615 We’ll have a very quick look at CouchDB’s bare-bones Application Pro‐
1616 gramming Interface (API) by using the command-line utility curl. Please
1617 note that this is not the only way of talking to CouchDB. We will show
1618 you plenty more throughout the rest of the documents. What’s interest‐
1619 ing about curl is that it gives you control over raw HTTP requests, and
1620 you can see exactly what is going on “underneath the hood” of your
1621 database.
1622 Make sure CouchDB is still running, and then do:
1624 curl http://127.0.0.1:5984/
1626 This issues a GET request to your newly installed CouchDB instance.
1628 The reply should look something like:
1630 {
1632 "couchdb": "Welcome",
1633 "version": "3.0.0",
1634 "git_sha": "83bdcf693",
1635 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
1636 "features": [
1637 "access-ready",
1638 "partitioned",
1639 "pluggable-storage-engines",
1640 "reshard",
1641 "scheduler"
1642 ],
1643 "vendor": {
1644 "name": "The Apache Software Foundation"
1645 }
1646 }
1647 Not all that spectacular. CouchDB is saying “hello” with the running
1649 version number.
1650 Next, we can get a list of databases:
1652 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1654 All we added to the previous request is the _all_dbs string, and our
1656 admin user name and password (set when installing CouchDB).
1657 The response should look like:
1659 ["_replicator","_users"]
1661 NOTE:
1663 In case this returns an empty Array for you, it means you haven’t
1664 finished installation correctly. Please refer to setup for further
1665 information on this.
1666 For the purposes of this example, we’ll not be showing the system
1668 databases past this point. In your installation, any time you GET
1669 /_all_dbs, you should see the system databases in the list, too.
1670 Oh, that’s right, we didn’t create any user databases yet!
1672 NOTE:
1674 The curl command issues GET requests by default. You can issue POST
1675 requests using curl -X POST. To make it easy to work with our termi‐
1676 nal history, we usually use the -X option even when issuing GET
1677 requests. If we want to send a POST next time, all we have to
1678 change is the method.
1679 HTTP does a bit more under the hood than you can see in the examples
1681 here. If you’re interested in every last detail that goes over the
1682 wire, pass in the -v option (e.g., curl -vX GET), which will show
1683 you the server curl tries to connect to, the request headers it
1684 sends, and response headers it receives back. Great for debugging!
1685 Let’s create a database:
1687 curl -X PUT http://admin:password@127.0.0.1:5984/baseball
1689 CouchDB will reply with:
1691 {"ok":true}
1693 Retrieving the list of databases again shows some useful results this
1695 time:
1696 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1698 ["baseball"]
1700 NOTE:
1702 We should mention JavaScript Object Notation (JSON) here, the data
1703 format CouchDB speaks. JSON is a lightweight data interchange format
1704 based on JavaScript syntax. Because JSON is natively compatible with
1705 JavaScript, your web browser is an ideal client for CouchDB.
1706 Brackets ([]) represent ordered lists, and curly braces ({}) repre‐
1708 sent key/value dictionaries. Keys must be strings, delimited by
1709 quotes ("), and values can be strings, numbers, booleans, lists, or
1710 key/value dictionaries. For a more detailed description of JSON, see
1711 Appendix E, JSON Primer.
1712 Let’s create another database:
1714 curl -X PUT http://admin:password@127.0.0.1:5984/baseball
1716 CouchDB will reply with:
1718 {"error":"file_exists","reason":"The database could not be created,
1720 the file already exists."}
1721 We already have a database with that name, so CouchDB will respond with
1723 an error. Let’s try again with a different database name:
1724 curl -X PUT http://admin:password@127.0.0.1:5984/plankton
1726 CouchDB will reply with:
1728 {"ok":true}
1730 Retrieving the list of databases yet again shows some useful results:
1732 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1734 CouchDB will respond with:
1736 ["baseball", "plankton"]
1738 To round things off, let’s delete the second database:
1740 curl -X DELETE http://admin:password@127.0.0.1:5984/plankton
1742 CouchDB will reply with:
1744 {"ok":true}
1746 The list of databases is now the same as it was before:
1748 curl -X GET http://admin:password@127.0.0.1:5984/_all_dbs
1750 CouchDB will respond with:
1752 ["baseball"]
1754 For brevity, we’ll skip working with documents, as the next section
1756 covers a different and potentially easier way of working with CouchDB
1757 that should provide experience with this. As we work through the exam‐
1758 ple, keep in mind that “under the hood” everything is being done by the
1759 application exactly as you have been doing here manually. Everything
1760 is done using GET, PUT, POST, and DELETE with a URI.
1761 Welcome to Fauxton
1763 After having seen CouchDB’s raw API, let’s get our feet wet by playing
1764 with Fauxton, the built-in administration interface. Fauxton provides
1765 full access to all of CouchDB’s features and makes it easy to work with
1766 some of the more complex ideas involved. With Fauxton we can create and
1767 destroy databases; view and edit documents; compose and run MapReduce
1768 views; and trigger replication between databases.
1769 To load Fauxton in your browser, visit:
1771 http://127.0.0.1:5984/_utils/
1773 and log in when prompted with your admin password.
1775 In later documents, we’ll focus on using CouchDB from server-side lan‐
1777 guages such as Ruby and Python. As such, this document is a great
1778 opportunity to showcase an example of natively serving up a dynamic web
1779 application using nothing more than CouchDB’s integrated web server,
1780 something you may wish to do with your own applications.
1781 The first thing we should do with a fresh installation of CouchDB is
1783 run the test suite to verify that everything is working properly. This
1784 assures us that any problems we may run into aren’t due to bothersome
1785 issues with our setup. By the same token, failures in the Fauxton test
1786 suite are a red flag, telling us to double-check our installation
1787 before attempting to use a potentially broken database server, saving
1788 us the confusion when nothing seems to be working quite like we expect!
1789 To validate your installation, click on the Verify link on the
1791 left-hand side, then press the green Verify Installation button. All
1792 tests should pass with a check mark. If any fail, re-check your instal‐
1793 lation steps.
1794 Your First Database and Document
1796 Creating a database in Fauxton is simple. From the overview page, click
1797 “Create Database.” When asked for a name, enter hello-world and click
1798 the Create button.
1799 After your database has been created, Fauxton will display a list of
1801 all its documents. This list will start out empty, so let’s create our
1802 first document. Click the plus sign next to “All Documents” and select
1803 the “New Doc” link. CouchDB will generate a UUID for you.
1804 For demoing purposes, having CouchDB assign a UUID is fine. When you
1806 write your first programs, we recommend assigning your own UUIDs. If
1807 you rely on the server to generate the UUID and you end up making two
1808 POST requests because the first POST request bombed out, you might gen‐
1809 erate two docs and never find out about the first one because only the
1810 second one will be reported back. Generating your own UUIDs makes sure
1811 that you’ll never end up with duplicate documents.
1812 Fauxton will display the newly created document, with its _id field. To
1814 create a new field, simply use the editor to write valid JSON. Add a
1815 new field by appending a comma to the _id value, then adding the text:
1816 "hello": "my new value"
1818 Click the green Create Document button to finalize creating the docu‐
1820 ment.
1821 You can experiment with other JSON values; e.g., [1, 2, "c"] or {"foo":
1823 "bar"}.
1824 You’ll notice that the document’s _rev has been added. We’ll go into
1826 more detail about this in later documents, but for now, the important
1827 thing to note is that _rev acts like a safety feature when saving a
1828 document. As long as you and CouchDB agree on the most recent _rev of a
1829 document, you can successfully save your changes.
1830 For clarity, you may want to display the contents of the document in
1832 the all document view. To enable this, from the upper-right corner of
1833 the window, select Options, then check the Include Docs option.
1834 Finally, press the Run Query button. The full document should be dis‐
1835 played along with the _id and _rev values.
1836 Running a Query Using MapReduce
1838 Traditional relational databases allow you to run any queries you like
1839 as long as your data is structured correctly. In contrast, CouchDB uses
1840 predefined map and reduce functions in a style known as MapReduce.
1841 These functions provide great flexibility because they can adapt to
1842 variations in document structure, and indexes for each document can be
1843 computed independently and in parallel. The combination of a map and a
1844 reduce function is called a view in CouchDB terminology.
1845 For experienced relational database programmers, MapReduce can take
1847 some getting used to. Rather than declaring which rows from which
1848 tables to include in a result set and depending on the database to
1849 determine the most efficient way to run the query, reduce queries are
1850 based on simple range requests against the indexes generated by your
1851 map functions.
1852 Map functions are called once with each document as the argument. The
1854 function can choose to skip the document altogether or emit one or more
1855 view rows as key/value pairs. Map functions may not depend on any
1856 information outside of the document. This independence is what allows
1857 CouchDB views to be generated incrementally and in parallel.
1858 CouchDB views are stored as rows that are kept sorted by key. This
1860 makes retrieving data from a range of keys efficient even when there
1861 are thousands or millions of rows. When writing CouchDB map functions,
1862 your primary goal is to build an index that stores related data under
1863 nearby keys.
1864 Before we can run an example MapReduce view, we’ll need some data to
1866 run it on. We’ll create documents carrying the price of various super‐
1867 market items as found at different shops. Let’s create documents for
1868 apples, oranges, and bananas. (Allow CouchDB to generate the _id and
1869 _rev fields.) Use Fauxton to create documents that have a final JSON
1870 structure that looks like this:
1871 {
1873 "_id": "00a271787f89c0ef2e10e88a0c0001f4",
1874 "_rev": "1-2628a75ac8c3abfffc8f6e30c9949fd6",
1875 "item": "apple",
1876 "prices": {
1877 "Fresh Mart": 1.59,
1878 "Price Max": 5.99,
1879 "Apples Express": 0.79
1880 }
1881 }
1882 OK, now that that’s done, let’s create the document for oranges:
1884 {
1886 "_id": "00a271787f89c0ef2e10e88a0c0003f0",
1887 "_rev": "1-e9680c5d9a688b4ff8dd68549e8e072c",
1888 "item": "orange",
1889 "prices": {
1890 "Fresh Mart": 1.99,
1891 "Price Max": 3.19,
1892 "Citrus Circus": 1.09
1893 }
1894 }
1895 And finally, the document for bananas:
1897 {
1899 "_id": "00a271787f89c0ef2e10e88a0c00048b",
1900 "_rev": "1-60e25d93dc12884676d037400a6fa189",
1901 "item": "banana",
1902 "prices": {
1903 "Fresh Mart": 1.99,
1904 "Price Max": 0.79,
1905 "Banana Montana": 4.22
1906 }
1907 }
1908 Imagine we’re catering a big luncheon, but the client is very
1910 price-sensitive. To find the lowest prices, we’re going to create our
1911 first view, which shows each fruit sorted by price. Click “All Docu‐
1912 ments” to return to the hello-world overview, and then from the “All
1913 Documents” plus sign, click “New View” to create a new view.
1914 Name the design document _design/myDesignDoc, and set the Index name to
1916 prices.
1917 Edit the map function, on the right, so that it looks like the follow‐
1919 ing:
1920 function(doc) {
1922 var shop, price, value;
1923 if (doc.item && doc.prices) {
1924 for (shop in doc.prices) {
1925 price = doc.prices[shop];
1926 value = [doc.item, shop];
1927 emit(price, value);
1928 }
1929 }
1930 }
1931 This is a JavaScript function that CouchDB runs for each of our docu‐
1933 ments as it computes the view. We’ll leave the reduce function blank
1934 for the time being.
1935 Click “Save Document and then Build Index” and you should see result
1937 rows, with the various items sorted by price. This map function could
1938 be even more useful if it grouped the items by type so that all the
1939 prices for bananas were next to each other in the result set. CouchDB’s
1940 key sorting system allows any valid JSON object as a key. In this case,
1941 we’ll emit an array of [item, price] so that CouchDB groups by item
1942 type and price.
1943 Let’s modify the view function (click the wrench icon next to the Views
1945 > prices Design Document on the left, then select Edit) so that it
1946 looks like this:
1947 function(doc) {
1949 var shop, price, key;
1950 if (doc.item && doc.prices) {
1951 for (shop in doc.prices) {
1952 price = doc.prices[shop];
1953 key = [doc.item, price];
1954 emit(key, shop);
1955 }
1956 }
1957 }
1958 Here, we first check that the document has the fields we want to use.
1960 CouchDB recovers gracefully from a few isolated map function failures,
1961 but when a map function fails regularly (due to a missing required
1962 field or other JavaScript exception), CouchDB shuts off its indexing to
1963 prevent any further resource usage. For this reason, it’s important to
1964 check for the existence of any fields before you use them. In this
1965 case, our map function will skip the first “hello world” document we
1966 created without emitting any rows or encountering any errors. The
1967 result of this query should now be displayed.
1968 Once we know we’ve got a document with an item type and some prices, we
1970 iterate over the item’s prices and emit key/values pairs. The key is an
1971 array of the item and the price, and forms the basis for CouchDB’s
1972 sorted index. In this case, the value is the name of the shop where the
1973 item can be found for the listed price.
1974 View rows are sorted by their keys – in this example, first by item,
1976 then by price. This method of complex sorting is at the heart of creat‐
1977 ing useful indexes with CouchDB.
1978 MapReduce can be challenging, especially if you’ve spent years working
1980 with relational databases. The important things to keep in mind are
1981 that map functions give you an opportunity to sort your data using any
1982 key you choose, and that CouchDB’s design is focused on providing fast,
1983 efficient access to data within a range of keys.
1984 Triggering Replication
1986 Fauxton can trigger replication between two local databases, between a
1987 local and remote database, or even between two remote databases. We’ll
1988 show you how to replicate data from one local database to another,
1989 which is a simple way of making backups of your databases as we’re
1990 working through the examples.
1991 First we’ll need to create an empty database to be the target of repli‐
1993 cation. Return to the Databases overview and create a database called
1994 hello-replication. Now click “Replication” in the sidebar and choose
1995 hello-world as the source and hello-replication as the target. Click
1996 “Replicate” to replicate your database.
1997 To view the result of your replication, click on the Databases tab
1999 again. You should see the hello-replication database has the same num‐
2000 ber of documents as the hello-world database, and it should take up
2001 roughly the same size as well.
2002 NOTE:
2004 For larger databases, replication can take much longer. It is impor‐
2005 tant to leave the browser window open while replication is taking
2006 place. As an alternative, you can trigger replication via curl or
2007 some other HTTP client that can handle long-running connections. If
2008 your client closes the connection before replication finishes,
2009 you’ll have to retrigger it. Luckily, CouchDB’s replication can
2010 take over from where it left off instead of starting from scratch.
2011 Wrapping Up
2013 Now that you’ve seen most of Fauxton’s features, you’ll be prepared to
2014 dive in and inspect your data as we build our example application in
2015 the next few documents. Fauxton’s pure JavaScript approach to managing
2016 CouchDB shows how it’s possible to build a fully featured web applica‐
2017 tion using only CouchDB’s HTTP API and integrated web server.
2018 But before we get there, we’ll have another look at CouchDB’s HTTP API
2020 – now with a magnifying glass. Let’s curl up on the couch and relax.
2021 The Core API
2023 This document explores the CouchDB in minute detail. It shows all the
2024 nitty-gritty and clever bits. We show you best practices and guide you
2025 around common pitfalls.
2026 We start out by revisiting the basic operations we ran in the previous
2028 document intro/tour, looking behind the scenes. We also show what Faux‐
2029 ton needs to do behind its user interface to give us the nice features
2030 we saw earlier.
2031 This document is both an introduction to the core CouchDB API as well
2033 as a reference. If you can’t remember how to run a particular request
2034 or why some parameters are needed, you can always come back here and
2035 look things up (we are probably the heaviest users of this document).
2036 While explaining the API bits and pieces, we sometimes need to take a
2038 larger detour to explain the reasoning for a particular request. This
2039 is a good opportunity for us to tell you why CouchDB works the way it
2040 does.
2041 The API can be subdivided into the following sections. We’ll explore
2043 them individually:
2044 · Server
2046 · Databases
2048 · Documents
2050 · Replication
2052 · Wrapping Up
2054 Server
2056 This one is basic and simple. It can serve as a sanity check to see if
2057 CouchDB is running at all. It can also act as a safety guard for
2058 libraries that require a certain version of CouchDB. We’re using the
2059 curl utility again:
2060 curl http://127.0.0.1:5984/
2062 CouchDB replies, all excited to get going:
2064 {
2066 "couchdb": "Welcome",
2067 "version": "3.0.0",
2068 "git_sha": "83bdcf693",
2069 "uuid": "56f16e7c93ff4a2dc20eb6acc7000b71",
2070 "features": [
2071 "access-ready",
2072 "partitioned",
2073 "pluggable-storage-engines",
2074 "reshard",
2075 "scheduler"
2076 ],
2077 "vendor": {
2078 "name": "The Apache Software Foundation"
2079 }
2080 }
2081 You get back a JSON string, that, if parsed into a native object or
2083 data structure of your programming language, gives you access to the
2084 welcome string and version information.
2085 This is not terribly useful, but it illustrates nicely the way CouchDB
2087 behaves. You send an HTTP request and you receive a JSON string in the
2088 HTTP response as a result.
2089 Databases
2091 Now let’s do something a little more useful: create databases. For the
2092 strict, CouchDB is a database management system (DMS). That means it
2093 can hold multiple databases. A database is a bucket that holds “related
2094 data”. We’ll explore later what that means exactly. In practice, the
2095 terminology is overlapping – often people refer to a DMS as “a data‐
2096 base” and also a database within the DMS as “a database.” We might fol‐
2097 low that slight oddity, so don’t get confused by it. In general, it
2098 should be clear from the context if we are talking about the whole of
2099 CouchDB or a single database within CouchDB.
2100 Now let’s make one! We want to store our favorite music albums, and we
2102 creatively give our database the name albums. Note that we’re now using
2103 the -X option again to tell curl to send a PUT request instead of the
2104 default GET request:
2105 curl -X PUT http://admin:password@127.0.0.1:5984/albums
2107 CouchDB replies:
2109 {"ok":true}
2111 That’s it. You created a database and CouchDB told you that all went
2113 well. What happens if you try to create a database that already
2114 exists? Let’s try to create that database again:
2115 curl -X PUT http://admin:password@127.0.0.1:5984/albums
2117 CouchDB replies:
2119 {"error":"file_exists","reason":"The database could not be created, the file already exists."}
2121 We get back an error. This is pretty convenient. We also learn a little
2123 bit about how CouchDB works. CouchDB stores each database in a single
2124 file. Very simple.
2125 Let’s create another database, this time with curl’s -v (for “verbose”)
2127 option. The verbose option tells curl to show us not only the essen‐
2128 tials – the HTTP response body – but all the underlying request and
2129 response details:
2130 curl -vX PUT http://admin:password@127.0.0.1:5984/albums-backup
2132 curl elaborates:
2134 * About to connect() to 127.0.0.1 port 5984 (#0)
2136 * Trying 127.0.0.1... connected
2137 * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
2138 > PUT /albums-backup HTTP/1.1
2139 > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
2140 > Host: 127.0.0.1:5984
2141 > Accept: */*
2142 >
2143 < HTTP/1.1 201 Created
2144 < Server: CouchDB (Erlang/OTP)
2145 < Date: Sun, 05 Jul 2009 22:48:28 GMT
2146 < Content-Type: text/plain;charset=utf-8
2147 < Content-Length: 12
2148 < Cache-Control: must-revalidate
2149 <
2150 {"ok":true}
2151 * Connection #0 to host 127.0.0.1 left intact
2152 * Closing connection #0
2153 What a mouthful. Let’s step through this line by line to understand
2155 what’s going on and find out what’s important. Once you’ve seen this
2156 output a few times, you’ll be able to spot the important bits more eas‐
2157 ily.
2158 * About to connect() to 127.0.0.1 port 5984 (#0)
2160 This is curl telling us that it is going to establish a TCP connection
2162 to the CouchDB server we specified in our request URI. Not at all
2163 important, except when debugging networking issues.
2164 * Trying 127.0.0.1... connected
2166 * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0)
2167 curl tells us it successfully connected to CouchDB. Again, not impor‐
2169 tant if you aren’t trying to find problems with your network.
2170 The following lines are prefixed with > and < characters. The > means
2172 the line was sent to CouchDB verbatim (without the actual >). The <
2173 means the line was sent back to curl by CouchDB.
2174 > PUT /albums-backup HTTP/1.1
2176 This initiates an HTTP request. Its method is PUT, the URI is
2178 /albums-backup, and the HTTP version is HTTP/1.1. There is also
2179 HTTP/1.0, which is simpler in some cases, but for all practical reasons
2180 you should be using HTTP/1.1.
2181 Next, we see a number of request headers. These are used to provide
2183 additional details about the request to CouchDB.
2184 > User-Agent: curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3
2186 The User-Agent header tells CouchDB which piece of client software is
2188 doing the HTTP request. We don’t learn anything new: it’s curl. This
2189 header is often useful in web development when there are known errors
2190 in client implementations that a server might want to prepare the
2191 response for. It also helps to determine which platform a user is on.
2192 This information can be used for technical and statistical reasons. For
2193 CouchDB, the User-Agent header is irrelevant.
2194 > Host: 127.0.0.1:5984
2196 The Host header is required by HTTP 1.1. It tells the server the host‐
2198 name that came with the request.
2199 > Accept: */*
2201 The Accept header tells CouchDB that curl accepts any media type.
2203 We’ll look into why this is useful a little later.
2204 >
2206 An empty line denotes that the request headers are now finished and the
2208 rest of the request contains data we’re sending to the server. In this
2209 case, we’re not sending any data, so the rest of the curl output is
2210 dedicated to the HTTP response.
2211 < HTTP/1.1 201 Created
2213 The first line of CouchDB’s HTTP response includes the HTTP version
2215 information (again, to acknowledge that the requested version could be
2216 processed), an HTTP status code, and a status code message. Different
2217 requests trigger different response codes. There’s a whole range of
2218 them telling the client (curl in our case) what effect the request had
2219 on the server. Or, if an error occurred, what kind of error. RFC 2616
2220 (the HTTP 1.1 specification) defines clear behavior for response codes.
2221 CouchDB fully follows the RFC.
2222 The 201 Created status code tells the client that the resource the
2224 request was made against was successfully created. No surprise here,
2225 but if you remember that we got an error message when we tried to cre‐
2226 ate this database twice, you now know that this response could include
2227 a different response code. Acting upon responses based on response
2228 codes is a common practice. For example, all response codes of 400 Bad
2229 Request or larger tell you that some error occurred. If you want to
2230 shortcut your logic and immediately deal with the error, you could just
2231 check a >= 400 response code.
2232 < Server: CouchDB (Erlang/OTP)
2234 The Server header is good for diagnostics. It tells us which CouchDB
2236 version and which underlying Erlang version we are talking to. In gen‐
2237 eral, you can ignore this header, but it is good to know it’s there if
2238 you need it.
2239 < Date: Sun, 05 Jul 2009 22:48:28 GMT
2241 The Date header tells you the time of the server. Since client and
2243 server time are not necessarily synchronized, this header is purely
2244 informational. You shouldn’t build any critical application logic on
2245 top of this!
2246 < Content-Type: text/plain;charset=utf-8
2248 The Content-Type header tells you which MIME type the HTTP response
2250 body is and its encoding. We already know CouchDB returns JSON strings.
2251 The appropriate Content-Type header is application/json. Why do we see
2252 text/plain? This is where pragmatism wins over purity. Sending an
2253 application/json Content-Type header will make a browser offer you the
2254 returned JSON for download instead of just displaying it. Since it is
2255 extremely useful to be able to test CouchDB from a browser, CouchDB
2256 sends a text/plain content type, so all browsers will display the JSON
2257 as text.
2258 NOTE:
2260 There are some extensions that make your browser JSON-aware, but
2261 they are not installed by default. For more information, look at the
2262 popular JSONView extension, available for both Firefox and Chrome.
2263 Do you remember the Accept request header and how it is set to */* to
2265 express interest in any MIME type? If you send Accept: application/json
2266 in your request, CouchDB knows that you can deal with a pure JSON
2267 response with the proper Content-Type header and will use it instead of
2268 text/plain.
2269 < Content-Length: 12
2271 The Content-Length header simply tells us how many bytes the response
2273 body has.
2274 < Cache-Control: must-revalidate
2276 This Cache-Control header tells you, or any proxy server between
2278 CouchDB and you, not to cache this response.
2279 <
2281 This empty line tells us we’re done with the response headers and what
2283 follows now is the response body.
2284 {"ok":true}
2286 We’ve seen this before.
2288 * Connection #0 to host 127.0.0.1 left intact
2290 * Closing connection #0
2291 The last two lines are curl telling us that it kept the TCP connection
2293 it opened in the beginning open for a moment, but then closed it after
2294 it received the entire response.
2295 Throughout the documents, we’ll show more requests with the -v option,
2297 but we’ll omit some of the headers we’ve seen here and include only
2298 those that are important for the particular request.
2299 Creating databases is all fine, but how do we get rid of one? Easy –
2301 just change the HTTP method:
2302 > curl -vX DELETE http://admin:password@127.0.0.1:5984/albums-backup
2304 This deletes a CouchDB database. The request will remove the file that
2306 the database contents are stored in. There is no “Are you sure?” safety
2307 net or any “Empty the trash” magic you’ve got to do to delete a data‐
2308 base. Use this command with care. Your data will be deleted without a
2309 chance to bring it back easily if you don’t have a backup copy.
2310 This section went knee-deep into HTTP and set the stage for discussing
2312 the rest of the core CouchDB API. Next stop: documents.
2313 Documents
2315 Documents are CouchDB’s central data structure. The idea behind a docu‐
2316 ment is, unsurprisingly, that of a real-world document – a sheet of
2317 paper such as an invoice, a recipe, or a business card. We already
2318 learned that CouchDB uses the JSON format to store documents. Let’s see
2319 how this storing works at the lowest level.
2320 Each document in CouchDB has an ID. This ID is unique per database. You
2322 are free to choose any string to be the ID, but for best results we
2323 recommend a UUID (or GUID), i.e., a Universally (or Globally) Unique
2324 IDentifier. UUIDs are random numbers that have such a low collision
2325 probability that everybody can make thousands of UUIDs a minute for
2326 millions of years without ever creating a duplicate. This is a great
2327 way to ensure two independent people cannot create two different docu‐
2328 ments with the same ID. Why should you care what somebody else is
2329 doing? For one, that somebody else could be you at a later time or on a
2330 different computer; secondly, CouchDB replication lets you share docu‐
2331 ments with others and using UUIDs ensures that it all works. But more
2332 on that later; let’s make some documents:
2333 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af -d '{"title":"There is Nothing Left to Lose","artist":"Foo Fighters"}'
2335 CouchDB replies:
2337 {"ok":true,"id":"6e1295ed6c29495e54cc05947f18c8af","rev":"1-2902191555"}
2339 The curl command appears complex, but let’s break it down. First, -X
2341 PUT tells curl to make a PUT request. It is followed by the URL that
2342 specifies your CouchDB IP address and port. The resource part of the
2343 URL /albums/6e1295ed6c29495e54cc05947f18c8af specifies the location of
2344 a document inside our albums database. The wild collection of numbers
2345 and characters is a UUID. This UUID is your document’s ID. Finally, the
2346 -d flag tells curl to use the following string as the body for the PUT
2347 request. The string is a simple JSON structure including title and
2348 artist attributes with their respective values.
2349 NOTE:
2351 If you don’t have a UUID handy, you can ask CouchDB to give you one
2352 (in fact, that is what we did just now without showing you). Simply
2353 send a GET /_uuids request:
2354 curl -X GET http://127.0.0.1:5984/_uuids
2356 CouchDB replies:
2358 {"uuids":["6e1295ed6c29495e54cc05947f18c8af"]}
2360 Voilà, a UUID. If you need more than one, you can pass in the
2362 ?count=10 HTTP parameter to request 10 UUIDs, or really, any number
2363 you need.
2364 To double-check that CouchDB isn’t lying about having saved your docu‐
2366 ment (it usually doesn’t), try to retrieve it by sending a GET request:
2367 curl -X GET http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af
2369 We hope you see a pattern here. Everything in CouchDB has an address, a
2371 URI, and you use the different HTTP methods to operate on these URIs.
2372 CouchDB replies:
2374 {"_id":"6e1295ed6c29495e54cc05947f18c8af","_rev":"1-2902191555","title":"There is Nothing Left to Lose","artist":"Foo Fighters"}
2376 This looks a lot like the document you asked CouchDB to save, which is
2378 good. But you should notice that CouchDB added two fields to your JSON
2379 structure. The first is _id, which holds the UUID we asked CouchDB to
2380 save our document under. We always know the ID of a document if it is
2381 included, which is very convenient.
2382 The second field is _rev. It stands for revision.
2384 Revisions
2386 If you want to change a document in CouchDB, you don’t tell it to go
2387 and find a field in a specific document and insert a new value.
2388 Instead, you load the full document out of CouchDB, make your changes
2389 in the JSON structure (or object, when you are doing actual program‐
2390 ming), and save the entire new revision (or version) of that document
2391 back into CouchDB. Each revision is identified by a new _rev value.
2392 If you want to update or delete a document, CouchDB expects you to
2394 include the _rev field of the revision you wish to change. When CouchDB
2395 accepts the change, it will generate a new revision number. This mecha‐
2396 nism ensures that, in case somebody else made a change without you
2397 knowing before you got to request the document update, CouchDB will not
2398 accept your update because you are likely to overwrite data you didn’t
2399 know existed. Or simplified: whoever saves a change to a document
2400 first, wins. Let’s see what happens if we don’t provide a _rev field
2401 (which is equivalent to providing a outdated value):
2402 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af \
2404 -d '{"title":"There is Nothing Left to Lose","artist":"Foo Fighters","year":"1997"}'
2405 CouchDB replies:
2407 {"error":"conflict","reason":"Document update conflict."}
2409 If you see this, add the latest revision number of your document to the
2411 JSON structure:
2412 curl -X PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af \
2414 -d '{"_rev":"1-2902191555","title":"There is Nothing Left to Lose","artist":"Foo Fighters","year":"1997"}'
2415 Now you see why it was handy that CouchDB returned that _rev when we
2417 made the initial request. CouchDB replies:
2418 {"ok":true,"id":"6e1295ed6c29495e54cc05947f18c8af","rev":"2-8aff9ee9d06671fa89c99d20a4b3ae"}
2420 CouchDB accepted your write and also generated a new revision number.
2422 The revision number is the MD5 hash of the transport representation of
2423 a document with an N- prefix denoting the number of times a document
2424 got updated. This is useful for replication. See replication/conflicts
2425 for more information.
2426 There are multiple reasons why CouchDB uses this revision system, which
2428 is also called Multi-Version Concurrency Control (MVCC). They all work
2429 hand-in-hand, and this is a good opportunity to explain some of them.
2430 One of the aspects of the HTTP protocol that CouchDB uses is that it is
2432 stateless. What does that mean? When talking to CouchDB you need to
2433 make requests. Making a request includes opening a network connection
2434 to CouchDB, exchanging bytes, and closing the connection. This is done
2435 every time you make a request. Other protocols allow you to open a con‐
2436 nection, exchange bytes, keep the connection open, exchange more bytes
2437 later – maybe depending on the bytes you exchanged at the beginning –
2438 and eventually close the connection. Holding a connection open for
2439 later use requires the server to do extra work. One common pattern is
2440 that for the lifetime of a connection, the client has a consistent and
2441 static view of the data on the server. Managing huge amounts of paral‐
2442 lel connections is a significant amount of work. HTTP connections are
2443 usually short-lived, and making the same guarantees is a lot easier.
2444 As a result, CouchDB can handle many more concurrent connections.
2445 Another reason CouchDB uses MVCC is that this model is simpler concep‐
2447 tually and, as a consequence, easier to program. CouchDB uses less code
2448 to make this work, and less code is always good because the ratio of
2449 defects per lines of code is static.
2450 The revision system also has positive effects on replication and stor‐
2452 age mechanisms, but we’ll explore these later in the documents.
2453 WARNING:
2455 The terms version and revision might sound familiar (if you are pro‐
2456 gramming without version control, stop reading this guide right now
2457 and start learning one of the popular systems). Using new versions
2458 for document changes works a lot like version control, but there’s
2459 an important difference: CouchDB does not guarantee that older ver‐
2460 sions are kept around. Don’t use the ``_rev`` token in CouchDB as a
2461 revision control system for your documents.
2462 Documents in Detail
2464 Now let’s have a closer look at our document creation requests with the
2465 curl -v flag that was helpful when we explored the database API ear‐
2466 lier. This is also a good opportunity to create more documents that we
2467 can use in later examples.
2468 We’ll add some more of our favorite music albums. Get a fresh UUID from
2470 the /_uuids resource. If you don’t remember how that works, you can
2471 look it up a few pages back.
2472 curl -vX PUT http://admin:password@127.0.0.1:5984/albums/70b50bfa0a4b3aed1f8aff9e92dc16a0 \
2474 -d '{"title":"Blackened Sky","artist":"Biffy Clyro","year":2002}'
2475 NOTE:
2477 By the way, if you happen to know more information about your
2478 favorite albums, don’t hesitate to add more properties. And don’t
2479 worry about not knowing all the information for all the albums.
2480 CouchDB’s schema-less documents can contain whatever you know. After
2481 all, you should relax and not worry about data.
2482 Now with the -v option, CouchDB’s reply (with only the important bits
2484 shown) looks like this:
2485 > PUT /albums/70b50bfa0a4b3aed1f8aff9e92dc16a0 HTTP/1.1
2487 >
2488 < HTTP/1.1 201 Created
2489 < Location: http://127.0.0.1:5984/albums/70b50bfa0a4b3aed1f8aff9e92dc16a0
2490 < ETag: "1-e89c99d29d06671fa0a4b3ae8aff9e"
2491 <
2492 {"ok":true,"id":"70b50bfa0a4b3aed1f8aff9e92dc16a0","rev":"1-e89c99d29d06671fa0a4b3ae8aff9e"}
2493 We’re getting back the 201 Created HTTP status code in the response
2495 headers, as we saw earlier when we created a database. The Location
2496 header gives us a full URL to our newly created document. And there’s a
2497 new header. An ETag in HTTP-speak identifies a specific version of a
2498 resource. In this case, it identifies a specific version (the first
2499 one) of our new document. Sound familiar? Yes, conceptually, an ETag is
2500 the same as a CouchDB document revision number, and it shouldn’t come
2501 as a surprise that CouchDB uses revision numbers for ETags. ETags are
2502 useful for caching infrastructures.
2503 Attachments
2505 CouchDB documents can have attachments just like an email message can
2506 have attachments. An attachment is identified by a name and includes
2507 its MIME type (or Content-Type) and the number of bytes the attachment
2508 contains. Attachments can be any data. It is easiest to think about
2509 attachments as files attached to a document. These files can be text,
2510 images, Word documents, music, or movie files. Let’s make one.
2511 Attachments get their own URL where you can upload data. Say we want to
2513 add the album artwork to the 6e1295ed6c29495e54cc05947f18c8af document
2514 (“There is Nothing Left to Lose”), and let’s also say the artwork is in
2515 a file artwork.jpg in the current directory:
2516 curl -vX PUT http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af/artwork.jpg?rev=2-2739352689 \
2518 --data-binary @artwork.jpg -H "Content-Type:image/jpg"
2519 NOTE:
2521 The --data-binary @ option tells curl to read a file’s contents into
2522 the HTTP request body. We’re using the -H option to tell CouchDB
2523 that we’re uploading a JPEG file. CouchDB will keep this information
2524 around and will send the appropriate header when requesting this
2525 attachment; in case of an image like this, a browser will render the
2526 image instead of offering you the data for download. This will come
2527 in handy later. Note that you need to provide the current revision
2528 number of the document you’re attaching the artwork to, just as if
2529 you would update the document. Because, after all, attaching some
2530 data is changing the document.
2531 You should now see your artwork image if you point your browser to
2533 http://127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af/artwork.jpg
2534 If you request the document again, you’ll see a new member:
2536 curl http://admin:password@127.0.0.1:5984/albums/6e1295ed6c29495e54cc05947f18c8af
2538 CouchDB replies:
2540 {
2542 "_id": "6e1295ed6c29495e54cc05947f18c8af",
2543 "_rev": "3-131533518",
2544 "title": "There is Nothing Left to Lose",
2545 "artist": "Foo Fighters",
2546 "year": "1997",
2547 "_attachments": {
2548 "artwork.jpg": {
2549 "stub": true,
2550 "content_type": "image/jpg",
2551 "length": 52450
2552 }
2553 }
2554 }
2555 _attachments is a list of keys and values where the values are JSON
2557 objects containing the attachment metadata. stub=true tells us that
2558 this entry is just the metadata. If we use the ?attachments=true HTTP
2559 option when requesting this document, we’d get a Base64 encoded string
2560 containing the attachment data.
2561 We’ll have a look at more document request options later as we explore
2563 more features of CouchDB, such as replication, which is the next topic.
2564 Replication
2566 CouchDB replication is a mechanism to synchronize databases. Much like
2567 rsync synchronizes two directories locally or over a network, replica‐
2568 tion synchronizes two databases locally or remotely.
2569 In a simple POST request, you tell CouchDB the source and the target of
2571 a replication and CouchDB will figure out which documents and new docu‐
2572 ment revisions are on source that are not yet on target, and will pro‐
2573 ceed to move the missing documents and revisions over.
2574 We’ll take an in-depth look at replication in the document replica‐
2576 tion/intro; in this document, we’ll just show you how to use it.
2577 First, we’ll create a target database. Note that CouchDB won’t automat‐
2579 ically create a target database for you, and will return a replication
2580 failure if the target doesn’t exist (likewise for the source, but that
2581 mistake isn’t as easy to make):
2582 curl -X PUT http://admin:password@127.0.0.1:5984/albums-replica
2584 Now we can use the database albums-replica as a replication target:
2586 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2588 -d '{"source":"http://127.0.0.1:5984/albums","target":"http://127.0.0.1:5984/albums-replica"}' \
2589 -H "Content-Type: application/json"
2590 NOTE:
2592 As of CouchDB 2.0.0, fully qualified URLs are required for both the
2593 replication source and target parameters.
2594 NOTE:
2596 CouchDB supports the option "create_target":true placed in the JSON
2597 POSTed to the _replicate URL. It implicitly creates the target data‐
2598 base if it doesn’t exist.
2599 CouchDB replies (this time we formatted the output so you can read it
2601 more easily):
2602 {
2604 "history": [
2605 {
2606 "start_last_seq": 0,
2607 "missing_found": 2,
2608 "docs_read": 2,
2609 "end_last_seq": 5,
2610 "missing_checked": 2,
2611 "docs_written": 2,
2612 "doc_write_failures": 0,
2613 "end_time": "Sat, 11 Jul 2009 17:36:21 GMT",
2614 "start_time": "Sat, 11 Jul 2009 17:36:20 GMT"
2615 }
2616 ],
2617 "source_last_seq": 5,
2618 "session_id": "924e75e914392343de89c99d29d06671",
2619 "ok": true
2620 }
2621 CouchDB maintains a session history of replications. The response for a
2623 replication request contains the history entry for this replication
2624 session. It is also worth noting that the request for replication will
2625 stay open until replication closes. If you have a lot of documents,
2626 it’ll take a while until they are all replicated and you won’t get back
2627 the replication response until all documents are replicated. It is
2628 important to note that replication replicates the database only as it
2629 was at the point in time when replication was started. So, any addi‐
2630 tions, modifications, or deletions subsequent to the start of replica‐
2631 tion will not be replicated.
2632 We’ll punt on the details again – the "ok": true at the end tells us
2634 all went well. If you now have a look at the albums-replica database,
2635 you should see all the documents that you created in the albums data‐
2636 base. Neat, eh?
2637 What you just did is called local replication in CouchDB terms. You
2639 created a local copy of a database. This is useful for backups or to
2640 keep snapshots of a specific state of your data around for later. You
2641 might want to do this if you are developing your applications but want
2642 to be able to roll back to a stable version of your code and data.
2643 There are more types of replication useful in other situations. The
2645 source and target members of our replication request are actually links
2646 (like in HTML) and so far we’ve seen links relative to the server we’re
2647 working on (hence local). You can also specify a remote database as the
2648 target:
2649 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2651 -d '{"source":"http://127.0.0.1:5984/albums","target":"http://example.org:5984/albums-replica"}' \
2652 -H "Content-Type:application/json"
2653 Using a local source and a remote target database is called push repli‐
2655 cation. We’re pushing changes to a remote server.
2656 NOTE:
2658 Since we don’t have a second CouchDB server around just yet, we’ll
2659 just use the absolute address of our single server, but you should
2660 be able to infer from this that you can put any remote server in
2661 there.
2662 This is great for sharing local changes with remote servers or buddies
2664 next door.
2665 You can also use a remote source and a local target to do a pull repli‐
2667 cation. This is great for getting the latest changes from a server that
2668 is used by others:
2669 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2671 -d '{"source":"http://example.org:5984/albums-replica","target":"http://127.0.0.1:5984/albums"}' \
2672 -H "Content-Type:application/json"
2673 Finally, you can run remote replication, which is mostly useful for
2675 management operations:
2676 curl -vX POST http://admin:password@127.0.0.1:5984/_replicate \
2678 -d '{"source":"http://example.org:5984/albums","target":"http://example.org:5984/albums-replica"}' \
2679 -H"Content-Type: application/json"
2680 NOTE:
2682 CouchDB and REST
2683 CouchDB prides itself on having a RESTful API, but these replication
2685 requests don’t look very RESTy to the trained eye. What’s up with
2686 that? While CouchDB’s core database, document, and attachment API
2687 are RESTful, not all of CouchDB’s API is. The replication API is one
2688 example. There are more, as we’ll see later in the documents.
2689 Why are there RESTful and non-RESTful APIs mixed up here? Have the
2691 developers been too lazy to go REST all the way? Remember, REST is
2692 an architectural style that lends itself to certain architectures
2693 (such as the CouchDB document API). But it is not a
2694 one-size-fits-all. Triggering an event like replication does not
2695 make a whole lot of sense in the REST world. It is more like a tra‐
2696 ditional remote procedure call. And there is nothing wrong with
2697 this.
2698 We very much believe in the “use the right tool for the job” philos‐
2700 ophy, and REST does not fit every job. For support, we refer to
2701 Leonard Richardson and Sam Ruby who wrote RESTful Web Services
2702 (O’Reilly), as they share our view.
2703 Wrapping Up
2705 This is still not the full CouchDB API, but we discussed the essentials
2706 in great detail. We’re going to fill in the blanks as we go. For now,
2707 we believe you’re ready to start building CouchDB applications.
2708 SEE ALSO:
2710 Complete HTTP API Reference:
2711 · Server API Reference
2713 · Database API Reference
2715 · Document API Reference
2717 · Replication API
2719
2721 Replication is an incremental one way process involving two databases
2722 (a source and a destination).
2723 The aim of replication is that at the end of the process, all active
2725 documents in the source database are also in the destination database
2726 and all documents that were deleted in the source database are also
2727 deleted in the destination database (if they even existed).
2728 The replication process only copies the last revision of a document, so
2730 all previous revisions that were only in the source database are not
2731 copied to the destination database.
2732 Introduction to Replication
2734 One of CouchDB’s strengths is the ability to synchronize two copies of
2735 the same database. This enables users to distribute data across several
2736 nodes or data centers, but also to move data more closely to clients.
2737 Replication involves a source and a destination database, which can be
2739 on the same or on different CouchDB instances. The aim of replication
2740 is that at the end of the process, all active documents in the source
2741 database are also in the destination database and all documents that
2742 were deleted in the source database are also deleted in the destination
2743 database (if they even existed).
2744 Transient and Persistent Replication
2746 There are two different ways to set up a replication. The first one
2747 that was introduced into CouchDB leads to a replication that could be
2748 called transient. Transient means that there are no documents backing
2749 up the replication. So after a restart of the CouchDB server the repli‐
2750 cation will disapear. Later, the _replicator database was introduced,
2751 which keeps documents containing your replication parameters. Such a
2752 replication can be called persistent. Transient replications were kept
2753 for backward compatibility. Both replications can have different repli‐
2754 cation states.
2755 Triggering, Stopping and Monitoring Replications
2757 A persistent replication is controlled through a document in the
2758 _replicator database, where each document describes one replication
2759 process (see replication-settings). For setting up a transient replica‐
2760 tion the api endpoint /_replicate can be used. A replication is trig‐
2761 gered by sending a JSON object either to the _replicate endpoint or
2762 storing it as a document into the _replicator database.
2763 If a replication is currently running its status can be inspected
2765 through the active tasks API (see api/server/active_tasks, replica‐
2766 tion-status and api/server/_scheduler/jobs).
2767 For document based-replications, api/server/_scheduler/docs can be used
2769 to get a complete state summary. This API is preferred as it will show
2770 the state of the replication document before it becomes a replication
2771 job.
2772 For transient replications there is no way to query their state when
2774 the job is finished.
2775 A replication can be stopped by deleting the document, or by updating
2777 it with its cancel property set to true.
2778 Replication Procedure
2780 During replication, CouchDB will compare the source and the destination
2781 database to determine which documents differ between the source and the
2782 destination database. It does so by following the changes on the source
2783 and comparing the documents to the destination. Changes are submitted
2784 to the destination in batches where they can introduce conflicts. Docu‐
2785 ments that already exist on the destination in the same revision are
2786 not transferred. As the deletion of documents is represented by a new
2787 revision, a document deleted on the source will also be deleted on the
2788 target.
2789 A replication task will finish once it reaches the end of the changes
2791 feed. If its continuous property is set to true, it will wait for new
2792 changes to appear until the task is canceled. Replication tasks also
2793 create checkpoint documents on the destination to ensure that a
2794 restarted task can continue from where it stopped, for example after it
2795 has crashed.
2796 When a replication task is initiated on the sending node, it is called
2798 push replication, if it is initiated by the receiving node, it is
2799 called pull replication.
2800 Master - Master replication
2802 One replication task will only transfer changes in one direction. To
2803 achieve master-master replication, it is possible to set up two repli‐
2804 cation tasks in opposite direction. When a change is replicated from
2805 database A to B by the first task, the second task from B to A will
2806 discover that the new change on B already exists in A and will wait for
2807 further changes.
2808 Controlling which Documents to Replicate
2810 There are three options for controlling which documents are replicated,
2811 and which are skipped:
2812 1. Defining documents as being local.
2814 2. Using selectorobj.
2816 3. Using filterfun.
2818 Local documents are never replicated (see api/local).
2820 selectorobj can be included in a replication document (see replica‐
2822 tion-settings). A selector object contains a query expression that is
2823 used to test whether a document should be replicated.
2824 filterfun can be used in a replication (see replication-settings). The
2826 replication task evaluates the filter function for each document in the
2827 changes feed. The document is only replicated if the filter returns
2828 true.
2829 NOTE:
2831 Using a selector provides performance benefits when compared with
2832 using a filterfun. You should use selectorobj where possible.
2833 NOTE:
2835 When using replication filters that depend on the document’s con‐
2836 tent, deleted documents may pose a problem, since the document
2837 passed to the filter will not contain any of the document’s content.
2838 This can be resolved by adding a _deleted:true field to the document
2839 instead of using the DELETE HTTP method, paired with the use of a
2840 validate document update handler to ensure the fields required for
2841 replication filters are always present. Take note, though, that the
2842 deleted document will still contain all of its data (including
2843 attachments)!
2844 Migrating Data to Clients
2846 Replication can be especially useful for bringing data closer to
2847 clients. PouchDB implements the replication algorithm of CouchDB in
2848 JavaScript, making it possible to make data from a CouchDB database
2849 available in an offline browser application, and synchronize changes
2850 back to CouchDB.
2851 Replicator Database
2853 Changed in version 2.1.0: Scheduling replicator was introduced. Repli‐
2854 cation states, by default are not written back to documents anymore.
2855 There are new replication job states and new API endpoints _sched‐
2856 uler/jobs and _scheduler/docs.
2857 The _replicator database works like any other in CouchDB, but documents
2860 added to it will trigger replications. Create (PUT or POST) a document
2861 to start replication. DELETE a replication document to cancel an ongo‐
2862 ing replication.
2863 These documents have exactly the same content as the JSON objects we
2865 used to POST to _replicate (fields source, target, create_target, con‐
2866 tinuous, doc_ids, filter, query_params, use_checkpoints, check‐
2867 point_interval).
2868 Replication documents can have a user defined _id (handy for finding a
2870 specific replication request later). Design Documents (and _local docu‐
2871 ments) added to the replicator database are ignored.
2872 The default replicator database is _replicator. Additional replicator
2874 databases can be created. To be recognized as such by the system, their
2875 database names should end with /_replicator.
2876 Basics
2878 Let’s say you POST the following document into _replicator:
2879 {
2881 "_id": "my_rep",
2882 "source": "http://myserver.com/foo",
2883 "target": "http://user:pass@localhost:5984/bar",
2884 "create_target": true,
2885 "continuous": true
2886 }
2887 In the couch log you’ll see 2 entries like these:
2889 [notice] 2017-04-05T17:16:19.646716Z node1@127.0.0.1 <0.29432.0> -------- Replication `"a81a78e822837e66df423d54279c15fe+continuous+create_target"` is using:
2891 4 worker processes
2892 a worker batch size of 500
2893 20 HTTP connections
2894 a connection timeout of 30000 milliseconds
2895 10 retries per request
2896 socket options are: [{keepalive,true},{nodelay,false}]
2897 [notice] 2017-04-05T17:16:19.646759Z node1@127.0.0.1 <0.29432.0> -------- Document `my_rep` triggered replication `a81a78e822837e66df423d54279c15fe+continuous+create_target`
2898 Replication state of this document can then be queried from
2900 http://adm:pass@localhost:5984/_scheduler/docs/_replicator/my_rep
2901 {
2903 "database": "_replicator",
2904 "doc_id": "my_rep",
2905 "error_count": 0,
2906 "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
2907 "info": {
2908 "revisions_checked": 113,
2909 "missing_revisions_found": 113,
2910 "docs_read": 113,
2911 "docs_written": 113,
2912 "changes_pending": 0,
2913 "doc_write_failures": 0,
2914 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2915 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2916 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
2917 },
2918 "last_updated": "2017-04-05T19:18:15Z",
2919 "node": "node1@127.0.0.1",
2920 "source_proxy": null,
2921 "target_proxy": null,
2922 "source": "http://myserver.com/foo/",
2923 "start_time": "2017-04-05T19:18:15Z",
2924 "state": "running",
2925 "target": "http://adm:*****@localhost:5984/bar/"
2926 }
2927 The state is running. That means replicator has scheduled this replica‐
2929 tion job to run. Replication document contents stay the same. Previ‐
2930 ously, before version 2.1, it was updated with the triggered state.
2931 The replication job will also appear in
2933 http://adm:pass@localhost:5984/_scheduler/jobs
2935 {
2937 "jobs": [
2938 {
2939 "database": "_replicator",
2940 "doc_id": "my_rep",
2941 "history": [
2942 {
2943 "timestamp": "2017-04-05T19:18:15Z",
2944 "type": "started"
2945 },
2946 {
2947 "timestamp": "2017-04-05T19:18:15Z",
2948 "type": "added"
2949 }
2950 ],
2951 "id": "a81a78e822837e66df423d54279c15fe+continuous+create_target",
2952 "info": {
2953 "changes_pending": 0,
2954 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2955 "doc_write_failures": 0,
2956 "docs_read": 113,
2957 "docs_written": 113,
2958 "missing_revisions_found": 113,
2959 "revisions_checked": 113,
2960 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
2961 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
2962 },
2963 "node": "node1@127.0.0.1",
2964 "pid": "<0.1174.0>",
2965 "source": "http://myserver.com/foo/",
2966 "start_time": "2017-04-05T19:18:15Z",
2967 "target": "http://adm:*****@localhost:5984/bar/",
2968 "user": null
2969 }
2970 ],
2971 "offset": 0,
2972 "total_rows": 1
2973 }
2974 _scheduler/jobs shows more information, such as a detailed history of
2976 state changes. If a persistent replication has not yet started, has
2977 failed, or is completed, information about its state can only be found
2978 in _scheduler/docs. Keep in mind that some replication documents could
2979 be invalid and could not become a replication job. Others might be
2980 delayed because they are fetching data from a slow source database.
2981 If there is an error, for example if the source database is missing,
2983 the replication job will crash and retry after a wait period. Each suc‐
2984 cessive crash will result in a longer waiting period.
2985 For example, POST-ing this document
2987 {
2989 "_id": "my_rep_crashing",
2990 "source": "http://myserver.com/missing",
2991 "target": "http://user:pass@localhost:5984/bar",
2992 "create_target": true,
2993 "continuous": true
2994 }
2995 when source database is missing, will result in periodic starts and
2997 crashes with an increasingly larger interval. The history list from
2998 _scheduler/jobs for this replication would look something like this:
2999 [
3001 {
3002 "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
3003 "timestamp": "2017-04-05T20:55:10Z",
3004 "type": "crashed"
3005 },
3006 {
3007 "timestamp": "2017-04-05T20:55:10Z",
3008 "type": "started"
3009 },
3010 {
3011 "reason": "db_not_found: could not open http://adm:*****@localhost:5984/missing/",
3012 "timestamp": "2017-04-05T20:47:10Z",
3013 "type": "crashed"
3014 },
3015 {
3016 "timestamp": "2017-04-05T20:47:10Z",
3017 "type": "started"
3018 }
3019 ]
3020 _scheduler/docs shows a shorter summary:
3022 {
3024 "database": "_replicator",
3025 "doc_id": "my_rep_crashing",
3026 "error_count": 6,
3027 "id": "cb78391640ed34e9578e638d9bb00e44+create_target",
3028 "info": {
3029 "error": "db_not_found: could not open http://adm:*****@localhost:5984/missing/"
3030 },
3031 "last_updated": "2017-04-05T20:55:10Z",
3032 "node": "node1@127.0.0.1",
3033 "source_proxy": null,
3034 "target_proxy": null,
3035 "source": "http://adm:*****@localhost:5984/missing/",
3036 "start_time": "2017-04-05T20:38:34Z",
3037 "state": "crashing",
3038 "target": "http://adm:*****@localhost:5984/bar/"
3039 }
3040 Repeated crashes are described as a crashing state. -ing suffix implies
3042 this is a temporary state. User at any moment could create the missing
3043 database and then replication job could return back to the normal.
3044 Documents describing the same replication
3046 Lets suppose 2 documents are added to the _replicator database in the
3047 following order:
3048 {
3050 "_id": "my_rep",
3051 "source": "http://myserver.com/foo",
3052 "target": "http://user:pass@localhost:5984/bar",
3053 "create_target": true,
3054 "continuous": true
3055 }
3056 and
3058 {
3060 "_id": "my_rep_dup",
3061 "source": "http://myserver.com/foo",
3062 "target": "http://user:pass@localhost:5984/bar",
3063 "create_target": true,
3064 "continuous": true
3065 }
3066 Both describe exactly the same replication (only their _ids differ).
3068 In this case document my_rep triggers the replication, while
3069 my_rep_dup` will fail. Inspecting _scheduler/docs explains exactly why
3070 it failed:
3071 {
3073 "database": "_replicator",
3074 "doc_id": "my_rep_dup",
3075 "error_count": 1,
3076 "id": null,
3077 "info": {
3078 "error": "Replication `a81a78e822837e66df423d54279c15fe+continuous+create_target` specified by document `my_rep_dup` already started, triggered by document `my_rep` from db `_replicator`"
3079 },
3080 "last_updated": "2017-04-05T21:41:51Z",
3081 "source": "http://myserver.com/foo/",
3082 "start_time": "2017-04-05T21:41:51Z",
3083 "state": "failed",
3084 "target": "http://adm:*****@localhost:5984/bar/"
3085 }
3086 Notice the state for this replication is failed. Unlike crashing,
3088 failed state is terminal. As long as both documents are present the
3089 replicator will not retry to run my_rep_dup replication. Another reason
3090 could be malformed documents. For example if worker process count is
3091 specified as a string ("worker_processes": "a few") instead of an inte‐
3092 ger, failure will occur.
3093 Replication Scheduler
3095 Once replication jobs are created they are managed by the scheduler.
3096 The scheduler is the replication component which periodically stops
3097 some jobs and starts others. This behavior makes it possible to have a
3098 larger number of jobs than the cluster could run simultaneously.
3099 Replication jobs which keep failing will be penalized and forced to
3100 wait. The wait time increases exponentially with each consecutive fail‐
3101 ure.
3102 When deciding which jobs to stop and which to start, the scheduler uses
3104 a round-robin algorithm to ensure fairness. Jobs which have been run‐
3105 ning the longest time will be stopped, and jobs which have been waiting
3106 the longest time will be started.
3107 NOTE:
3109 Non-continuous (normal) replication are treated differently once
3110 they start running. See Normal vs Continuous Replications section
3111 for more information.
3112 The behavior of the scheduler can configured via max_jobs, interval and
3114 max_churn options. See Replicator configuration section for additional
3115 information.
3116 Replication states
3118 Replication jobs during their life-cycle pass through various states.
3119 This is a diagram of all the states and transitions between them:
3120 [image: Replication state diagram] [image] Replication state dia‐
3121 gram.UNINDENT
3122 Blue and yellow shapes represent replication job states.
3124 Trapezoidal shapes represent external APIs, that’s how users interact
3126 with the replicator. Writing documents to _replicator is the pre‐
3127 ferred way of creating replications, but posting to the _replicate
3128 HTTP endpoint is also supported.
3129 Six-sided shapes are internal API boundaries. They are optional for
3131 this diagram and are only shown as additional information to help
3132 clarify how the replicator works. There are two processing stages:
3133 the first is where replication documents are parsed and become repli‐
3134 cation jobs, and the second is the scheduler itself. The scheduler
3135 runs replication jobs, periodically stopping and starting some. Jobs
3136 posted via the _replicate endpoint bypass the first component and go
3137 straight to the scheduler.
3138 States descriptions
3140 Before explaining the details of each state, it is worth noticing that
3141 color and shape of each state in the diagram:
3142 Blue vs yellow partitions states into “healthy” and “unhealthy”,
3144 respectively. Unhealthy states indicate something has gone wrong and it
3145 might need user’s attention.
3146 Rectangle vs oval separates “terminal” states from “non-terminal” ones.
3148 Terminal states are those which will not transition to other states any
3149 more. Informally, jobs in a terminal state will not be retried and
3150 don’t consume memory or CPU resources.
3151 · Initializing: Indicates replicator has noticed the change from the
3153 replication document. Jobs should transition quickly through this
3154 state. Being stuck here for a while could mean there is an inter‐
3155 nal error.
3156 · Failed: Replication document could not be processed and turned
3158 into a valid replication job for the scheduler. This state is ter‐
3159 minal and requires user intervention to fix the problem. A typical
3160 reason for ending up in this state is a malformed document. For
3161 example, specifying an integer for a parameter which accepts a
3162 boolean. Another reason for failure could be specifying a dupli‐
3163 cate replication. A duplicate replication is a replication with
3164 identical parameters but a different document ID.
3165 · Error: Replication document update could not be turned into a
3167 replication job. Unlike the Failed state, this one is temporary,
3168 and replicator will keep retrying periodically. There is an expo‐
3169 nential backoff applied in case of consecutive failures. The main
3170 reason this state exists is to handle filtered replications with
3171 custom user functions. Filter function content is needed in order
3172 to calculate the replication ID. A replication job could not be
3173 created until the function code is retrieved. Because retrieval
3174 happens over the network, temporary failures have to be handled.
3175 · Running: Replication job is running normally. This means, there
3177 might be a change feed open, and if changes are noticed, they
3178 would be processed and posted to the target. Job is still consid‐
3179 ered Running even if its workers are currently not streaming
3180 changes from source to target and are just waiting on the change
3181 feed. Continuous replications will most likely end up in this
3182 state.
3183 · Pending: Replication job is not running and is waiting its turn.
3185 This state is reached when the number of replication jobs added to
3186 the scheduler exceeds replicator.max_jobs. In that case scheduler
3187 will periodically stop and start subsets of jobs trying to give
3188 each one a fair chance at making progress.
3189 · Crashing: Replication job has been successfully added to the
3191 replication scheduler. However an error was encountered during the
3192 last run. Error could be a network failure, a missing source data‐
3193 base, a permissions error, etc. Repeated consecutive crashes
3194 result in an exponential backoff. This state is considered tempo‐
3195 rary (non-terminal) and replication jobs will be periodically
3196 retried. Maximum backoff interval is around a day or so.
3197 · Completed: This is a terminal, successful state for non-continuous
3199 replications. Once in this state the replication is “forgotten” by
3200 the scheduler and it doesn’t consume any more CPU or memory
3201 resources. Continuous replication jobs will never reach this
3202 state.
3203 Normal vs Continuous Replications
3205 Normal (non-continuous) replications once started will be allowed to
3206 run to completion. That behavior is to preserve their semantics of
3207 replicating a snapshot of the source database to the target. For exam‐
3208 ple if new documents are added to the source after the replication are
3209 started, those updates should not show up on the target database.
3210 Stopping and restring a normal replication would violate that con‐
3211 straint.
3212 WARNING:
3214 When there is a mix of continuous and normal replications, once nor‐
3215 mal replication are scheduled to run, they might temporarily starve
3216 continuous replication jobs.
3217 However, normal replications will still be stopped and rescheduled if
3219 an operator reduces the value for the maximum number of replications.
3220 This is so that if an operator decides replications are overwhelming a
3221 node that it has the ability to recover. Any stopped replications will
3222 be resubmitted to the queue to be rescheduled.
3223 Compatibility Mode
3225 Previous version of CouchDB replicator wrote state updates back to
3226 replication documents. In cases where user code programmatically read
3227 those states, there is compatibility mode enabled via a configuration
3228 setting:
3229 [replicator]
3231 update_docs = true
3232 In this mode replicator will continue to write state updates to the
3234 documents.
3235 To effectively disable the scheduling behavior, which periodically stop
3237 and starts jobs, set max_jobs configuration setting to a large number.
3238 For example:
3239 [replicator]
3241 max_jobs = 9999999
3242 See Replicator configuration section for other replicator configuration
3244 options.
3245 Canceling replications
3247 To cancel a replication simply DELETE the document which triggered the
3248 replication. To update a replication, for example, change the number of
3249 worker or the source, simply update the document with new data. If
3250 there is extra application-specific data in the replication documents,
3251 that data is ignored by the replicator.
3252 Server restart
3254 When CouchDB is restarted, it checks its _replicator databases and
3255 restarts replications described by documents if they are not already in
3256 in a completed or failed state. If they are, they are ignored.
3257 Clustering
3259 In a cluster, replication jobs are balanced evenly among all the nodes
3260 nodes such that a replication job runs on only one node at a time.
3261 Every time there is a cluster membership change, that is when nodes are
3263 added or removed, as it happens in a rolling reboot, replicator appli‐
3264 cation will notice the change, rescan all the document and running
3265 replication, and re-evaluate their cluster placement in light of the
3266 new set of live nodes. This mechanism also provides replication
3267 fail-over in case a node fails. Replication jobs started from replica‐
3268 tion documents (but not those started from _replicate HTTP endpoint)
3269 will automatically migrate one of the live nodes.
3270 Additional Replicator Databases
3272 Imagine replicator database (_replicator) has these two documents which
3273 represent pull replications from servers A and B:
3274 {
3276 "_id": "rep_from_A",
3277 "source": "http://aserver.com:5984/foo",
3278 "target": "http://user:pass@localhost:5984/foo_a",
3279 "continuous": true
3280 }
3281 {
3283 "_id": "rep_from_B",
3284 "source": "http://bserver.com:5984/foo",
3285 "target": "http://user:pass@localhost:5984/foo_b",
3286 "continuous": true
3287 }
3288 Now without stopping and restarting CouchDB, add another replicator
3290 database. For example another/_replicator:
3291 $ curl -X PUT http://user:pass@localhost:5984/another%2F_replicator/
3293 {"ok":true}
3294 NOTE:
3296 A / character in a database name, when used in a URL, should be
3297 escaped.
3298 Then add a replication document to the new replicator database:
3300 {
3302 "_id": "rep_from_X",
3303 "source": "http://xserver.com:5984/foo",
3304 "target": "http://user:pass@localhost:5984/foo_x",
3305 "continuous": true
3306 }
3307 From now on, there are three replications active in the system: two
3309 replications from A and B, and a new one from X.
3310 Then remove the additional replicator database:
3312 $ curl -X DELETE http://user:pass@localhost:5984/another%2F_replicator/
3314 {"ok":true}
3315 After this operation, replication pulling from server X will be stopped
3317 and the replications in the _replicator database (pulling from servers
3318 A and B) will continue.
3319 Replicating the replicator database
3321 Imagine you have in server C a replicator database with the two follow‐
3322 ing pull replication documents in it:
3323 {
3325 "_id": "rep_from_A",
3326 "source": "http://aserver.com:5984/foo",
3327 "target": "http://user:pass@localhost:5984/foo_a",
3328 "continuous": true
3329 }
3330 {
3332 "_id": "rep_from_B",
3333 "source": "http://bserver.com:5984/foo",
3334 "target": "http://user:pass@localhost:5984/foo_b",
3335 "continuous": true
3336 }
3337 Now you would like to have the same pull replications going on in
3339 server D, that is, you would like to have server D pull replicating
3340 from servers A and B. You have two options:
3341 · Explicitly add two documents to server’s D replicator database
3343 · Replicate server’s C replicator database into server’s D replicator
3345 database
3346 Both alternatives accomplish exactly the same goal.
3348 Delegations
3350 Replication documents can have a custom user_ctx property. This prop‐
3351 erty defines the user context under which a replication runs. For the
3352 old way of triggering a replication (POSTing to /_replicate/), this
3353 property is not needed. That’s because information about the authenti‐
3354 cated user is readily available during the replication, which is not
3355 persistent in that case. Now, with the replicator database, the problem
3356 is that information about which user is starting a particular replica‐
3357 tion is only present when the replication document is written. The
3358 information in the replication document and the replication itself are
3359 persistent, however. This implementation detail implies that in the
3360 case of a non-admin user, a user_ctx property containing the user’s
3361 name and a subset of their roles must be defined in the replication
3362 document. This is enforced by the document update validation function
3363 present in the default design document of the replicator database. The
3364 validation function also ensures that non-admin users are unable to set
3365 the value of the user context’s name property to anything other than
3366 their own user name. The same principle applies for roles.
3367 For admins, the user_ctx property is optional, and if it’s missing it
3369 defaults to a user context with name null and an empty list of roles,
3370 which means design documents won’t be written to local targets. If
3371 writing design documents to local targets is desired, the role _admin
3372 must be present in the user context’s list of roles.
3373 Also, for admins the user_ctx property can be used to trigger a repli‐
3375 cation on behalf of another user. This is the user context that will be
3376 passed to local target database document validation functions.
3377 NOTE:
3379 The user_ctx property only has effect for local endpoints.
3380 Example delegated replication document:
3382 {
3384 "_id": "my_rep",
3385 "source": "http://bserver.com:5984/foo",
3386 "target": "http://user:pass@localhost:5984/bar",
3387 "continuous": true,
3388 "user_ctx": {
3389 "name": "joe",
3390 "roles": ["erlanger", "researcher"]
3391 }
3392 }
3393 As stated before, the user_ctx property is optional for admins, while
3395 being mandatory for regular (non-admin) users. When the roles property
3396 of user_ctx is missing, it defaults to the empty list [].
3397 Selector Objects
3399 Including a Selector Object in the replication document enables you to
3400 use a query expression to determine if a document should be included in
3401 the replication.
3402 The selector specifies fields in the document, and provides an expres‐
3404 sion to evaluate with the field content or other data. If the expres‐
3405 sion resolves to true, the document is replicated.
3406 The selector object must:
3408 · Be structured as valid JSON.
3410 · Contain a valid query expression.
3412 The syntax for a selector is the same as the selectorsyntax used for
3414 _find.
3415 Using a selector is significantly more efficient than using a
3417 JavaScript filter function, and is the recommended option if filtering
3418 on document attributes only.
3419 Replication and conflict model
3421 Let’s take the following example to illustrate replication and conflict
3422 handling.
3423 · Alice has a document containing Bob’s business card;
3425 · She synchronizes it between her desktop PC and her laptop;
3427 · On the desktop PC, she updates Bob’s E-mail address; Without syncing
3429 again, she updates Bob’s mobile number on the laptop;
3430 · Then she replicates the two to each other again.
3432 So on the desktop the document has Bob’s new E-mail address and his old
3434 mobile number, and on the laptop it has his old E-mail address and his
3435 new mobile number.
3436 The question is, what happens to these conflicting updated documents?
3438 CouchDB replication
3440 CouchDB works with JSON documents inside databases. Replication of
3441 databases takes place over HTTP, and can be either a “pull” or a
3442 “push”, but is unidirectional. So the easiest way to perform a full
3443 sync is to do a “push” followed by a “pull” (or vice versa).
3444 So, Alice creates v1 and sync it. She updates to v2a on one side and
3446 v2b on the other, and then replicates. What happens?
3447 The answer is simple: both versions exist on both sides!
3449 DESKTOP LAPTOP
3451 +---------+
3452 | /db/bob | INITIAL
3453 | v1 | CREATION
3454 +---------+
3455 +---------+ +---------+
3457 | /db/bob | -----------------> | /db/bob | PUSH
3458 | v1 | | v1 |
3459 +---------+ +---------+
3460 +---------+ +---------+ INDEPENDENT
3462 | /db/bob | | /db/bob | LOCAL
3463 | v2a | | v2b | EDITS
3464 +---------+ +---------+
3465 +---------+ +---------+
3467 | /db/bob | -----------------> | /db/bob | PUSH
3468 | v2a | | v2a |
3469 +---------+ | v2b |
3470 +---------+
3471 +---------+ +---------+
3473 | /db/bob | <----------------- | /db/bob | PULL
3474 | v2a | | v2a |
3475 | v2b | | v2b |
3476 +---------+ +---------+
3477 After all, this is not a file system, so there’s no restriction that
3479 only one document can exist with the name /db/bob. These are just “con‐
3480 flicting” revisions under the same name.
3481 Because the changes are always replicated, the data is safe. Both
3483 machines have identical copies of both documents, so failure of a hard
3484 drive on either side won’t lose any of the changes.
3485 Another thing to notice is that peers do not have to be configured or
3487 tracked. You can do regular replications to peers, or you can do
3488 one-off, ad-hoc pushes or pulls. After the replication has taken place,
3489 there is no record kept of which peer any particular document or revi‐
3490 sion came from.
3491 So the question now is: what happens when you try to read /db/bob? By
3493 default, CouchDB picks one arbitrary revision as the “winner”, using a
3494 deterministic algorithm so that the same choice will be made on all
3495 peers. The same happens with views: the deterministically-chosen winner
3496 is the only revision fed into your map function.
3497 Let’s say that the winner is v2a. On the desktop, if Alice reads the
3499 document she’ll see v2a, which is what she saved there. But on the lap‐
3500 top, after replication, she’ll also see only v2a. It could look as if
3501 the changes she made there have been lost - but of course they have
3502 not, they have just been hidden away as a conflicting revision. But
3503 eventually she’ll need these changes merged into Bob’s business card,
3504 otherwise they will effectively have been lost.
3505 Any sensible business-card application will, at minimum, have to
3507 present the conflicting versions to Alice and allow her to create a new
3508 version incorporating information from them all. Ideally it would merge
3509 the updates itself.
3510 Conflict avoidance
3512 When working on a single node, CouchDB will avoid creating conflicting
3513 revisions by returning a 409 Conflict error. This is because, when you
3514 PUT a new version of a document, you must give the _rev of the previous
3515 version. If that _rev has already been superseded, the update is
3516 rejected with a 409 Conflict response.
3517 So imagine two users on the same node are fetching Bob’s business card,
3519 updating it concurrently, and writing it back:
3520 USER1 -----------> GET /db/bob
3522 <----------- {"_rev":"1-aaa", ...}
3523 USER2 -----------> GET /db/bob
3525 <----------- {"_rev":"1-aaa", ...}
3526 USER1 -----------> PUT /db/bob?rev=1-aaa
3528 <----------- {"_rev":"2-bbb", ...}
3529 USER2 -----------> PUT /db/bob?rev=1-aaa
3531 <----------- 409 Conflict (not saved)
3532 User2’s changes are rejected, so it’s up to the app to fetch /db/bob
3534 again, and either:
3535 1. apply the same changes as were applied to the earlier revision, and
3537 submit a new PUT
3538 2. redisplay the document so the user has to edit it again
3540 3. just overwrite it with the document being saved before (which is not
3542 advisable, as user1’s changes will be silently lost)
3543 So when working in this mode, your application still has to be able to
3545 handle these conflicts and have a suitable retry strategy, but these
3546 conflicts never end up inside the database itself.
3547 Revision tree
3549 When you update a document in CouchDB, it keeps a list of the previous
3550 revisions. In the case where conflicting updates are introduced, this
3551 history branches into a tree, where the current conflicting revisions
3552 for this document form the tips (leaf nodes) of this tree:
3553 ,--> r2a
3555 r1 --> r2b
3556 `--> r2c
3557 Each branch can then extend its history - for example if you read revi‐
3559 sion r2b and then PUT with ?rev=r2b then you will make a new revision
3560 along that particular branch.
3561 ,--> r2a -> r3a -> r4a
3563 r1 --> r2b -> r3b
3564 `--> r2c -> r3c
3565 Here, (r4a, r3b, r3c) are the set of conflicting revisions. The way you
3567 resolve a conflict is to delete the leaf nodes along the other
3568 branches. So when you combine (r4a+r3b+r3c) into a single merged docu‐
3569 ment, you would replace r4a and delete r3b and r3c.
3570 ,--> r2a -> r3a -> r4a -> r5a
3572 r1 --> r2b -> r3b -> (r4b deleted)
3573 `--> r2c -> r3c -> (r4c deleted)
3574 Note that r4b and r4c still exist as leaf nodes in the history tree,
3576 but as deleted docs. You can retrieve them but they will be marked
3577 "_deleted":true.
3578 When you compact a database, the bodies of all the non-leaf documents
3580 are discarded. However, the list of historical _revs is retained, for
3581 the benefit of later conflict resolution in case you meet any old
3582 replicas of the database at some time in future. There is “revision
3583 pruning” to stop this getting arbitrarily large.
3584 Working with conflicting documents
3586 The basic :get:`/{doc}/{docid}` operation will not show you any infor‐
3587 mation about conflicts. You see only the deterministically-chosen win‐
3588 ner, and get no indication as to whether other conflicting revisions
3589 exist or not:
3590 {
3592 "_id":"test",
3593 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3594 "hello":"bar"
3595 }
3596 If you do GET /db/test?conflicts=true, and the document is in a con‐
3598 flict state, then you will get the winner plus a _conflicts member con‐
3599 taining an array of the revs of the other, conflicting revision(s). You
3600 can then fetch them individually using subsequent GET /db/test?rev=xxxx
3601 operations:
3602 {
3604 "_id":"test",
3605 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3606 "hello":"bar",
3607 "_conflicts":[
3608 "2-65db2a11b5172bf928e3bcf59f728970",
3609 "2-5bc3c6319edf62d4c624277fdd0ae191"
3610 ]
3611 }
3612 If you do GET /db/test?open_revs=all then you will get all the leaf
3614 nodes of the revision tree. This will give you all the current con‐
3615 flicts, but will also give you leaf nodes which have been deleted (i.e.
3616 parts of the conflict history which have since been resolved). You can
3617 remove these by filtering out documents with "_deleted":true:
3618 [
3620 {"ok":{"_id":"test","_rev":"2-5bc3c6319edf62d4c624277fdd0ae191","hello":"foo"}},
3621 {"ok":{"_id":"test","_rev":"2-65db2a11b5172bf928e3bcf59f728970","hello":"baz"}},
3622 {"ok":{"_id":"test","_rev":"2-b91bb807b4685080c6a651115ff558f5","hello":"bar"}}
3623 ]
3624 The "ok" tag is an artifact of open_revs, which also lets you list
3626 explicit revisions as a JSON array, e.g. open_revs=[rev1,rev2,rev3]. In
3627 this form, it would be possible to request a revision which is now
3628 missing, because the database has been compacted.
3629 NOTE:
3631 The order of revisions returned by open_revs=all is NOT related to
3632 the deterministic “winning” algorithm. In the above example, the
3633 winning revision is 2-b91b… and happens to be returned last, but in
3634 other cases it can be returned in a different position.
3635 Once you have retrieved all the conflicting revisions, your application
3637 can then choose to display them all to the user. Or it could attempt to
3638 merge them, write back the merged version, and delete the conflicting
3639 versions - that is, to resolve the conflict permanently.
3640 As described above, you need to update one revision and delete all the
3642 conflicting revisions explicitly. This can be done using a single POST
3643 to _bulk_docs, setting "_deleted":true on those revisions you wish to
3644 delete.
3645 Multiple document API
3647 Finding conflicted documents with Mango
3648 New in version 2.2.0.
3649 CouchDB’s Mango system allows easy querying of documents with con‐
3652 flicts, returning the full body of each document as well.
3653 Here’s how to use it to find all conflicts in a database:
3655 $ curl -X POST http://127.0.0.1/dbname/_find \
3657 -d '{"selector": {"_conflicts": { "$exists": true}}, "conflicts": true}' \
3658 -Hcontent-type:application/json
3659 {"docs": [
3661 {"_id":"doc","_rev":"1-3975759ccff3842adf690a5c10caee42","a":2,"_conflicts":["1-23202479633c2b380f79507a776743d5"]}
3662 ],
3663 "bookmark": "g1AAAABheJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYozA1kgKQ6YVA5QkBFMgKSVDHWNjI0MjEzMLc2MjZONkowtDNLMLU0NzBPNzc3MTYxTTLOysgCY2ReV"}
3664 The bookmark value can be used to navigate through additional pages of
3666 results if necessary. Mango by default only returns 25 results per
3667 request.
3668 If you expect to run this query often, be sure to create a Mango sec‐
3670 ondary index to speed the query:
3671 $ curl -X POST http://127.0.0.1/dbname/_index \
3673 -d '{"index":{"fields": ["_conflicts"]}}' \
3674 -Hcontent-type:application/json
3675 Of course, the selector can be enhanced to filter documents on addi‐
3677 tional keys in the document. Be sure to add those keys to your sec‐
3678 ondary index as well, or a full database scan will be triggered.
3679 Finding conflicted documents using the _all_docs index
3681 You can fetch multiple documents at once using include_docs=true on a
3682 view. However, a conflicts=true request is ignored; the “doc” part of
3683 the value never includes a _conflicts member. Hence you would need to
3684 do another query to determine for each document whether it is in a con‐
3685 flicting state:
3686 $ curl 'http://127.0.0.1:5984/conflict_test/_all_docs?include_docs=true&conflicts=true'
3688 {
3690 "total_rows":1,
3691 "offset":0,
3692 "rows":[
3693 {
3694 "id":"test",
3695 "key":"test",
3696 "value":{"rev":"2-b91bb807b4685080c6a651115ff558f5"},
3697 "doc":{
3698 "_id":"test",
3699 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3700 "hello":"bar"
3701 }
3702 }
3703 ]
3704 }
3705 $ curl 'http://127.0.0.1:5984/conflict_test/test?conflicts=true'
3707 {
3709 "_id":"test",
3710 "_rev":"2-b91bb807b4685080c6a651115ff558f5",
3711 "hello":"bar",
3712 "_conflicts":[
3713 "2-65db2a11b5172bf928e3bcf59f728970",
3714 "2-5bc3c6319edf62d4c624277fdd0ae191"
3715 ]
3716 }
3717 View map functions
3719 Views only get the winning revision of a document. However they do also
3720 get a _conflicts member if there are any conflicting revisions. This
3721 means you can write a view whose job is specifically to locate docu‐
3722 ments with conflicts. Here is a simple map function which achieves
3723 this:
3724 function(doc) {
3726 if (doc._conflicts) {
3727 emit(null, [doc._rev].concat(doc._conflicts));
3728 }
3729 }
3730 which gives the following output:
3732 {
3734 "total_rows":1,
3735 "offset":0,
3736 "rows":[
3737 {
3738 "id":"test",
3739 "key":null,
3740 "value":[
3741 "2-b91bb807b4685080c6a651115ff558f5",
3742 "2-65db2a11b5172bf928e3bcf59f728970",
3743 "2-5bc3c6319edf62d4c624277fdd0ae191"
3744 ]
3745 }
3746 ]
3747 }
3748 If you do this, you can have a separate “sweep” process which periodi‐
3750 cally scans your database, looks for documents which have conflicts,
3751 fetches the conflicting revisions, and resolves them.
3752 Whilst this keeps the main application simple, the problem with this
3754 approach is that there will be a window between a conflict being intro‐
3755 duced and it being resolved. From a user’s viewpoint, this may appear
3756 that the document they just saved successfully may suddenly lose their
3757 changes, only to be resurrected some time later. This may or may not be
3758 acceptable.
3759 Also, it’s easy to forget to start the sweeper, or not to implement it
3761 properly, and this will introduce odd behaviour which will be hard to
3762 track down.
3763 CouchDB’s “winning” revision algorithm may mean that information drops
3765 out of a view until a conflict has been resolved. Consider Bob’s busi‐
3766 ness card again; suppose Alice has a view which emits mobile numbers,
3767 so that her telephony application can display the caller’s name based
3768 on caller ID. If there are conflicting documents with Bob’s old and new
3769 mobile numbers, and they happen to be resolved in favour of Bob’s old
3770 number, then the view won’t be able to recognise his new one. In this
3771 particular case, the application might have preferred to put informa‐
3772 tion from both the conflicting documents into the view, but this cur‐
3773 rently isn’t possible.
3774 Suggested algorithm to fetch a document with conflict resolution:
3776 1. Get document via GET docid?conflicts=true request
3778 2. For each member in the _conflicts array call GET docid?rev=xxx. If
3780 any errors occur at this stage, restart from step 1. (There could
3781 be a race where someone else has already resolved this conflict and
3782 deleted that rev)
3783 3. Perform application-specific merging
3785 4. Write _bulk_docs with an update to the first rev and deletes of the
3787 other revs.
3788 This could either be done on every read (in which case you could
3790 replace all calls to GET in your application with calls to a library
3791 which does the above), or as part of your sweeper code.
3792 And here is an example of this in Ruby using the low-level RestClient:
3794 require 'rubygems'
3796 require 'rest_client'
3797 require 'json'
3798 DB="http://127.0.0.1:5984/conflict_test"
3799 # Write multiple documents
3801 def writem(docs)
3802 JSON.parse(RestClient.post("#{DB}/_bulk_docs", {
3803 "docs" => docs,
3804 }.to_json))
3805 end
3806 # Write one document, return the rev
3808 def write1(doc, id=nil, rev=nil)
3809 doc['_id'] = id if id
3810 doc['_rev'] = rev if rev
3811 writem([doc]).first['rev']
3812 end
3813 # Read a document, return *all* revs
3815 def read1(id)
3816 retries = 0
3817 loop do
3818 # FIXME: escape id
3819 res = [JSON.parse(RestClient.get("#{DB}/#{id}?conflicts=true"))]
3820 if revs = res.first.delete('_conflicts')
3821 begin
3822 revs.each do |rev|
3823 res << JSON.parse(RestClient.get("#{DB}/#{id}?rev=#{rev}"))
3824 end
3825 rescue
3826 retries += 1
3827 raise if retries >= 5
3828 next
3829 end
3830 end
3831 return res
3832 end
3833 end
3834 # Create DB
3836 RestClient.delete DB rescue nil
3837 RestClient.put DB, {}.to_json
3838 # Write a document
3840 rev1 = write1({"hello"=>"xxx"},"test")
3841 p read1("test")
3842 # Make three conflicting versions
3844 write1({"hello"=>"foo"},"test",rev1)
3845 write1({"hello"=>"bar"},"test",rev1)
3846 write1({"hello"=>"baz"},"test",rev1)
3847 res = read1("test")
3849 p res
3850 # Now let's replace these three with one
3852 res.first['hello'] = "foo+bar+baz"
3853 res.each_with_index do |r,i|
3854 unless i == 0
3855 r.replace({'_id'=>r['_id'], '_rev'=>r['_rev'], '_deleted'=>true})
3856 end
3857 end
3858 writem(res)
3859 p read1("test")
3861 An application written this way never has to deal with a PUT 409, and
3863 is automatically multi-master capable.
3864 You can see that it’s straightforward enough when you know what you’re
3866 doing. It’s just that CouchDB doesn’t currently provide a convenient
3867 HTTP API for “fetch all conflicting revisions”, nor “PUT to supersede
3868 these N revisions”, so you need to wrap these yourself. At the time of
3869 writing, there are no known client-side libraries which provide support
3870 for this.
3871 Merging and revision history
3873 Actually performing the merge is an application-specific function. It
3874 depends on the structure of your data. Sometimes it will be easy: e.g.
3875 if a document contains a list which is only ever appended to, then you
3876 can perform a union of the two list versions.
3877 Some merge strategies look at the changes made to an object, compared
3879 to its previous version. This is how Git’s merge function works.
3880 For example, to merge Bob’s business card versions v2a and v2b, you
3882 could look at the differences between v1 and v2b, and then apply these
3883 changes to v2a as well.
3884 With CouchDB, you can sometimes get hold of old revisions of a docu‐
3886 ment. For example, if you fetch /db/bob?rev=v2b&revs_info=true you’ll
3887 get a list of the previous revision ids which ended up with revision
3888 v2b. Doing the same for v2a you can find their common ancestor revi‐
3889 sion. However if the database has been compacted, the content of that
3890 document revision will have been lost. revs_info will still show that
3891 v1 was an ancestor, but report it as “missing”:
3892 BEFORE COMPACTION AFTER COMPACTION
3894 ,-> v2a v2a
3896 v1
3897 `-> v2b v2b
3898 So if you want to work with diffs, the recommended way is to store
3900 those diffs within the new revision itself. That is: when you replace
3901 v1 with v2a, include an extra field or attachment in v2a which says
3902 which fields were changed from v1 to v2a. This unfortunately does mean
3903 additional book-keeping for your application.
3904 Comparison with other replicating data stores
3906 The same issues arise with other replicating systems, so it can be
3907 instructive to look at these and see how they compare with CouchDB.
3908 Please feel free to add other examples.
3909 Unison
3911 Unison is a bi-directional file synchronisation tool. In this case, the
3912 business card would be a file, say bob.vcf.
3913 When you run unison, changes propagate both ways. If a file has changed
3915 on one side but not the other, the new replaces the old. Unison main‐
3916 tains a local state file so that it knows whether a file has changed
3917 since the last successful replication.
3918 In our example it has changed on both sides. Only one file called
3920 bob.vcf can exist within the file system. Unison solves the problem by
3921 simply ducking out: the user can choose to replace the remote version
3922 with the local version, or vice versa (both of which would lose data),
3923 but the default action is to leave both sides unchanged.
3924 From Alice’s point of view, at least this is a simple solution. When‐
3926 ever she’s on the desktop she’ll see the version she last edited on the
3927 desktop, and whenever she’s on the laptop she’ll see the version she
3928 last edited there.
3929 But because no replication has actually taken place, the data is not
3931 protected. If her laptop hard drive dies, she’ll lose all her changes
3932 made on the laptop; ditto if her desktop hard drive dies.
3933 It’s up to her to copy across one of the versions manually (under a
3935 different filename), merge the two, and then finally push the merged
3936 version to the other side.
3937 Note also that the original file (version v1) has been lost at this
3939 point. So it’s not going to be known from inspection alone whether v2a
3940 or v2b has the most up-to-date E-mail address for Bob, or which version
3941 has the most up-to-date mobile number. Alice has to remember which one
3942 she entered last.
3943 Git
3945 Git is a well-known distributed source control system. Like Unison, Git
3946 deals with files. However, Git considers the state of a whole set of
3947 files as a single object, the “tree”. Whenever you save an update, you
3948 create a “commit” which points to both the updated tree and the previ‐
3949 ous commit(s), which in turn point to the previous tree(s). You there‐
3950 fore have a full history of all the states of the files. This history
3951 forms a branch, and a pointer is kept to the tip of the branch, from
3952 which you can work backwards to any previous state. The “pointer” is an
3953 SHA1 hash of the tip commit.
3954 If you are replicating with one or more peers, a separate branch is
3956 made for each of those peers. For example, you might have:
3957 master -- my local branch
3959 remotes/foo/master -- branch on peer 'foo'
3960 remotes/bar/master -- branch on peer 'bar'
3961 In the regular workflow, replication is a “pull”, importing changes
3963 from a remote peer into the local repository. A “pull” does two things:
3964 first “fetch” the state of the peer into the remote tracking branch for
3965 that peer; and then attempt to “merge” those changes into the local
3966 branch.
3967 Now let’s consider the business card. Alice has created a Git repo con‐
3969 taining bob.vcf, and cloned it across to the other machine. The
3970 branches look like this, where AAAAAAAA is the SHA1 of the commit:
3971 ---------- desktop ---------- ---------- laptop ----------
3973 master: AAAAAAAA master: AAAAAAAA
3974 remotes/laptop/master: AAAAAAAA remotes/desktop/master: AAAAAAAA
3975 Now she makes a change on the desktop, and commits it into the desktop
3977 repo; then she makes a different change on the laptop, and commits it
3978 into the laptop repo:
3979 ---------- desktop ---------- ---------- laptop ----------
3981 master: BBBBBBBB master: CCCCCCCC
3982 remotes/laptop/master: AAAAAAAA remotes/desktop/master: AAAAAAAA
3983 Now on the desktop she does git pull laptop. First, the remote objects
3985 are copied across into the local repo and the remote tracking branch is
3986 updated:
3987 ---------- desktop ---------- ---------- laptop ----------
3989 master: BBBBBBBB master: CCCCCCCC
3990 remotes/laptop/master: CCCCCCCC remotes/desktop/master: AAAAAAAA
3991 NOTE:
3993 The repo still contains AAAAAAAA because commits BBBBBBBB and CCCCC‐
3994 CCC point to it.
3995 Then Git will attempt to merge the changes in. Knowing that the parent
3997 commit to CCCCCCCC is AAAAAAAA, it takes a diff between AAAAAAAA and
3998 CCCCCCCC and tries to apply it to BBBBBBBB.
3999 If this is successful, then you’ll get a new version with a merge com‐
4001 mit:
4002 ---------- desktop ---------- ---------- laptop ----------
4004 master: DDDDDDDD master: CCCCCCCC
4005 remotes/laptop/master: CCCCCCCC remotes/desktop/master: AAAAAAAA
4006 Then Alice has to logon to the laptop and run git pull desktop. A simi‐
4008 lar process occurs. The remote tracking branch is updated:
4009 ---------- desktop ---------- ---------- laptop ----------
4011 master: DDDDDDDD master: CCCCCCCC
4012 remotes/laptop/master: CCCCCCCC remotes/desktop/master: DDDDDDDD
4013 Then a merge takes place. This is a special case: CCCCCCCC is one of
4015 the parent commits of DDDDDDDD, so the laptop can fast forward update
4016 from CCCCCCCC to DDDDDDDD directly without having to do any complex
4017 merging. This leaves the final state as:
4018 ---------- desktop ---------- ---------- laptop ----------
4020 master: DDDDDDDD master: DDDDDDDD
4021 remotes/laptop/master: CCCCCCCC remotes/desktop/master: DDDDDDDD
4022 Now this is all and good, but you may wonder how this is relevant when
4024 thinking about CouchDB.
4025 First, note what happens in the case when the merge algorithm fails.
4027 The changes are still propagated from the remote repo into the local
4028 one, and are available in the remote tracking branch. So, unlike Uni‐
4029 son, you know the data is protected. It’s just that the local working
4030 copy may fail to update, or may diverge from the remote version. It’s
4031 up to you to create and commit the combined version yourself, but you
4032 are guaranteed to have all the history you might need to do this.
4033 Note that while it is possible to build new merge algorithms into Git,
4035 the standard ones are focused on line-based changes to source code.
4036 They don’t work well for XML or JSON if it’s presented without any line
4037 breaks.
4038 The other interesting consideration is multiple peers. In this case you
4040 have multiple remote tracking branches, some of which may match your
4041 local branch, some of which may be behind you, and some of which may be
4042 ahead of you (i.e. contain changes that you haven’t yet merged):
4043 master: AAAAAAAA
4045 remotes/foo/master: BBBBBBBB
4046 remotes/bar/master: CCCCCCCC
4047 remotes/baz/master: AAAAAAAA
4048 Note that each peer is explicitly tracked, and therefore has to be
4050 explicitly created. If a peer becomes stale or is no longer needed,
4051 it’s up to you to remove it from your configuration and delete the
4052 remote tracking branch. This is different from CouchDB, which doesn’t
4053 keep any peer state in the database.
4054 Another difference between CouchDB and Git is that it maintains all
4056 history back to time zero - Git compaction keeps diffs between all
4057 those versions in order to reduce size, but CouchDB discards them. If
4058 you are constantly updating a document, the size of a Git repo would
4059 grow forever. It is possible (with some effort) to use “history rewrit‐
4060 ing” to make Git forget commits earlier than a particular one.
4061 What is the CouchDB replication protocol? Is it like Git?
4063 Author Jason Smith
4064 Date 2011-01-29
4066 Source StackOverflow
4068 Key points
4070 If you know Git, then you know how Couch replication works. Replicating
4072 is very similar to pushing or pulling with distributed source managers
4073 like Git.
4074 CouchDB replication does not have its own protocol. A replicator simply
4076 connects to two DBs as a client, then reads from one and writes to the
4077 other. Push replication is reading the local data and updating the
4078 remote DB; pull replication is vice versa.
4079 · Fun fact 1: The replicator is actually an independent Erlang applica‐
4081 tion, in its own process. It connects to both couches, then reads
4082 records from one and writes them to the other.
4083 · Fun fact 2: CouchDB has no way of knowing who is a normal client and
4085 who is a replicator (let alone whether the replication is push or
4086 pull). It all looks like client connections. Some of them read
4087 records. Some of them write records.
4088 Everything flows from the data model
4090 The replication algorithm is trivial, uninteresting. A trained monkey
4092 could design it. It’s simple because the cleverness is the data model,
4093 which has these useful characteristics:
4094 1. Every record in CouchDB is completely independent of all others.
4096 That sucks if you want to do a JOIN or a transaction, but it’s awe‐
4097 some if you want to write a replicator. Just figure out how to
4098 replicate one record, and then repeat that for each record.
4099 2. Like Git, records have a linked-list revision history. A record’s
4101 revision ID is the checksum of its own data. Subsequent revision IDs
4102 are checksums of: the new data, plus the revision ID of the previ‐
4103 ous.
4104 3. In addition to application data ({"name": "Jason", "awesome":
4106 true}), every record stores the evolutionary time line of all previ‐
4107 ous revision IDs leading up to itself.
4108 · Exercise: Take a moment of quiet reflection. Consider any two dif‐
4110 ferent records, A and B. If A’s revision ID appears in B’s time
4111 line, then B definitely evolved from A. Now consider Git’s
4112 fast-forward merges. Do you hear that? That is the sound of your
4113 mind being blown.
4114 4. Git isn’t really a linear list. It has forks, when one parent has
4116 multiple children. CouchDB has that too.
4117 · Exercise: Compare two different records, A and B. A’s revision ID
4119 does not appear in B’s time line; however, one revision ID, C, is
4120 in both A’s and B’s time line. Thus A didn’t evolve from B. B
4121 didn’t evolve from A. But rather, A and B have a common ancestor
4122 C. In Git, that is a “fork.” In CouchDB, it’s a “conflict.”
4123 · In Git, if both children go on to develop their time lines inde‐
4125 pendently, that’s cool. Forks totally support that.
4126 · In CouchDB, if both children go on to develop their time lines
4128 independently, that cool too. Conflicts totally support that.
4129 · Fun fact 3: CouchDB “conflicts” do not correspond to Git “con‐
4131 flicts.” A Couch conflict is a divergent revision history, what
4132 Git calls a “fork.” For this reason the CouchDB community pro‐
4133 nounces “conflict” with a silent n: “co-flicked.”
4134 5. Git also has merges, when one child has multiple parents. CouchDB
4136 sort of has that too.
4137 · In the data model, there is no merge. The client simply marks one
4139 time line as deleted and continues to work with the only extant
4140 time line.
4141 · In the application, it feels like a merge. Typically, the client
4143 merges the data from each time line in an application-specific
4144 way. Then it writes the new data to the time line. In Git, this
4145 is like copying and pasting the changes from branch A into branch
4146 B, then committing to branch B and deleting branch A. The data was
4147 merged, but there was no git merge.
4148 · These behaviors are different because, in Git, the time line
4150 itself is important; but in CouchDB, the data is important and the
4151 time line is incidental—it’s just there to support replication.
4152 That is one reason why CouchDB’s built-in revisioning is inappro‐
4153 priate for storing revision data like a wiki page.
4154 Final notes
4156 At least one sentence in this writeup (possibly this one) is complete
4158 BS.
4159 CouchDB Replication Protocol
4161 Version
4162 3
4163 The CouchDB Replication Protocol is a protocol for synchronising JSON
4165 documents between 2 peers over HTTP/1.1 by using the public CouchDB
4166 REST API and is based on the Apache CouchDB MVCC Data model.
4167 Preface
4169 Language
4170 The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”,
4171 “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this
4172 document are to be interpreted as described in RFC 2119.
4173 Goals
4175 The primary goal of this specification is to describe the CouchDB
4176 Replication Protocol under the hood.
4177 The secondary goal is to provide enough detailed information about the
4179 protocol to make it easy to build tools on any language and platform
4180 that can synchronize data with CouchDB.
4181 Definitions
4183 JSON: JSON is a text format for the serialization of structured data.
4184 It is described in ECMA-262 and RFC 4627.
4185 URI: A URI is defined by RFC 3986. It can be a URL as defined in RFC
4187 1738.
4188 ID: An identifier (could be a UUID) as described in RFC 4122.
4190 Revision:
4192 A MVCC token value of following pattern: N-sig where N is ALWAYS
4193 a positive integer and sig is the Document signature (custom).
4194 Don’t mix it up with the revision in version control systems!
4195 Leaf Revision:
4197 The last Document Revision in a series of changes. Documents may
4198 have multiple Leaf Revisions (aka Conflict Revisions) due to
4199 concurrent updates.
4200 Document:
4202 A document is a JSON object with an ID and Revision defined in
4203 _id and _rev fields respectively. A Document’s ID MUST be unique
4204 within the Database where it is stored.
4205 Database:
4207 A collection of Documents with a unique URI.
4208 Changes Feed:
4210 A stream of Document-changing events (create, update, delete)
4211 for the specified Database.
4212 Sequence ID:
4214 An ID provided by the Changes Feed. It MUST be incremental, but
4215 MAY NOT always be an integer.
4216 Source:
4218 Database from where the Documents are replicated.
4219 Target:
4221 Database where the Documents are replicated to.
4222 Replication:
4224 The one-way directed synchronization process of Source and Tar‐
4225 get endpoints.
4226 Checkpoint:
4228 Intermediate Recorded Sequence ID used for Replication recovery.
4229 Replicator:
4231 A service or an application which initiates and runs Replica‐
4232 tion.
4233 Filter Function:
4235 A special function of any programming language that is used to
4236 filter Documents during Replication (see filterfun)
4237 Filter Function Name:
4239 An ID of a Filter Function that may be used as a symbolic refer‐
4240 ence (aka callback function) to apply the related Filter Func‐
4241 tion to Replication.
4242 Filtered Replication:
4244 Replication of Documents from Source to Target using a Filter
4245 Function.
4246 Full Replication:
4248 Replication of all Documents from Source to Target.
4249 Push Replication:
4251 Replication process where Source is a local endpoint and Target
4252 is remote.
4253 Pull Replication:
4255 Replication process where Source is a remote endpoint and Target
4256 is local.
4257 Continuous Replication:
4259 Replication that “never stops”: after processing all events from
4260 the Changes Feed, the Replicator doesn’t close the connection,
4261 but awaits new change events from the Source. The connection is
4262 kept alive by periodic heartbeats.
4263 Replication Log:
4265 A special Document that holds Replication history (recorded
4266 Checkpoints and a few more statistics) between Source and Tar‐
4267 get.
4268 Replication ID:
4270 A unique value that unambiguously identifies the Replication
4271 Log.
4272 Replication Protocol Algorithm
4274 The CouchDB Replication Protocol is not magical, but an agreement on
4275 usage of the public CouchDB HTTP REST API to enable Documents to be
4276 replicated from Source to Target.
4277 The reference implementation, written in Erlang, is provided by the
4279 couch_replicator module in Apache CouchDB.
4280 It is RECOMMENDED that one follow this algorithm specification, use the
4282 same HTTP endpoints, and run requests with the same parameters to pro‐
4283 vide a completely compatible implementation. Custom Replicator imple‐
4284 mentations MAY use different HTTP API endpoints and request parameters
4285 depending on their local specifics and they MAY implement only part of
4286 the Replication Protocol to run only Push or Pull Replication. However,
4287 while such solutions could also run the Replication process, they loose
4288 compatibility with the CouchDB Replicator.
4289 Verify Peers
4291 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4292 ' Verify Peers: '
4293 ' '
4294 ' 404 Not Found +--------------------------------+ '
4295 ' +----------------------- | Check Source Existence | '
4296 ' | +--------------------------------+ '
4297 ' | | HEAD /source | '
4298 ' | +--------------------------------+ '
4299 ' | | '
4300 ' | | 200 OK '
4301 ' | v '
4302 ' | +--------------------------------+ '
4303 ' | | Check Target Existence | ----+ '
4304 ' | +--------------------------------+ | '
4305 ' | | HEAD /target | | '
4306 ' | +--------------------------------+ | '
4307 ' | | | '
4308 ' | | 404 Not Found | '
4309 ' v v | '
4310 ' +-------+ No +--------------------------------+ | '
4311 ' | Abort | <----------------- | Create Target? | | '
4312 ' +-------+ +--------------------------------+ | '
4313 ' ^ | | '
4314 ' | | Yes | '
4315 ' | v | '
4316 ' | Failure +--------------------------------+ | '
4317 ' +----------------------- | Create Target | | '
4318 ' +--------------------------------+ | '
4319 ' | PUT /target | | '
4320 ' +--------------------------------+ | '
4321 ' | | '
4322 ' | 201 Created 200 OK | '
4323 ' | | '
4324 + - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - | - +
4325 | |
4326 + - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - | - +
4327 ' Get Peers Information: | | '
4328 ' +------------------------------------+ '
4329 ' | '
4330 ' v '
4331 ' +--------------------------------+ '
4332 ' | Get Source Information | '
4333 ' +--------------------------------+ '
4334 ' '
4335 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4336 The Replicator MUST ensure that both Source and Target exist by using
4338 HEAD /{db} requests.
4339 Check Source Existence
4341 Request:
4342 HEAD /source HTTP/1.1
4344 Host: localhost:5984
4345 User-Agent: CouchDB
4346 Response:
4348 HTTP/1.1 200 OK
4350 Cache-Control: must-revalidate
4351 Content-Type: application/json
4352 Date: Sat, 05 Oct 2013 08:50:39 GMT
4353 Server: CouchDB (Erlang/OTP)
4354 Check Target Existence
4356 Request:
4357 HEAD /target HTTP/1.1
4359 Host: localhost:5984
4360 User-Agent: CouchDB
4361 Response:
4363 HTTP/1.1 200 OK
4365 Cache-Control: must-revalidate
4366 Content-Type: application/json
4367 Date: Sat, 05 Oct 2013 08:51:11 GMT
4368 Server: CouchDB (Erlang/OTP)
4369 Create Target?
4371 In case of a non-existent Target, the Replicator MAY make a PUT /{db}
4372 request to create the Target:
4373 Request:
4374 PUT /target HTTP/1.1
4376 Accept: application/json
4377 Host: localhost:5984
4378 User-Agent: CouchDB
4379 Response:
4381 HTTP/1.1 201 Created
4383 Content-Length: 12
4384 Content-Type: application/json
4385 Date: Sat, 05 Oct 2013 08:58:41 GMT
4386 Server: CouchDB (Erlang/OTP)
4387 {
4389 "ok": true
4390 }
4391 However, the Replicator’s PUT request MAY NOT succeeded due to insuffi‐
4393 cient privileges (which are granted by the provided credential) and so
4394 receive a 401 Unauthorized or a 403 Forbidden error. Such errors SHOULD
4395 be expected and well handled:
4396 HTTP/1.1 500 Internal Server Error
4398 Cache-Control: must-revalidate
4399 Content-Length: 108
4400 Content-Type: application/json
4401 Date: Fri, 09 May 2014 13:50:32 GMT
4402 Server: CouchDB (Erlang OTP)
4403 {
4405 "error": "unauthorized",
4406 "reason": "unauthorized to access or create database http://localhost:5984/target"
4407 }
4408 Abort
4410 In case of a non-existent Source or Target, Replication SHOULD be
4411 aborted with an HTTP error response:
4412 HTTP/1.1 500 Internal Server Error
4414 Cache-Control: must-revalidate
4415 Content-Length: 56
4416 Content-Type: application/json
4417 Date: Sat, 05 Oct 2013 08:55:29 GMT
4418 Server: CouchDB (Erlang OTP)
4419 {
4421 "error": "db_not_found",
4422 "reason": "could not open source"
4423 }
4424 Get Peers Information
4426 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
4427 ' Verify Peers: '
4428 ' +------------------------+ '
4429 ' | Check Target Existence | '
4430 ' +------------------------+ '
4431 ' | '
4432 ' | 200 OK '
4433 ' | '
4434 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4435 |
4436 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4437 ' Get Peers Information: | '
4438 ' v '
4439 ' +------------------------+ '
4440 ' | Get Source Information | '
4441 ' +------------------------+ '
4442 ' | GET /source | '
4443 ' +------------------------+ '
4444 ' | '
4445 ' | 200 OK '
4446 ' v '
4447 ' +------------------------+ '
4448 ' | Get Target Information | '
4449 ' +------------------------+ '
4450 ' | GET /target | '
4451 ' +------------------------+ '
4452 ' | '
4453 ' | 200 OK '
4454 ' | '
4455 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4456 |
4457 + - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - -+
4458 ' Find Common Ancestry: | '
4459 ' | '
4460 ' v '
4461 ' +-------------------------+ '
4462 ' | Generate Replication ID | '
4463 ' +-------------------------+ '
4464 ' '
4465 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -+
4466 The Replicator retrieves basic information both from Source and Target
4468 using GET /{db} requests. The GET response MUST contain JSON objects
4469 with the following mandatory fields:
4470 · instance_start_time (string): Always "0". (Returned for legacy rea‐
4472 sons.)
4473 · update_seq (number / string): The current database Sequence ID.
4475 Any other fields are optional. The information that the Replicator
4477 needs is the update_seq field: this value will be used to define a tem‐
4478 porary (because Database data is subject to change) upper bound for
4479 changes feed listening and statistic calculating to show proper Repli‐
4480 cation progress.
4481 Get Source Information
4483 Request:
4484 GET /source HTTP/1.1
4486 Accept: application/json
4487 Host: localhost:5984
4488 User-Agent: CouchDB
4489 Response:
4491 HTTP/1.1 200 OK
4493 Cache-Control: must-revalidate
4494 Content-Length: 256
4495 Content-Type: application/json
4496 Date: Tue, 08 Oct 2013 07:53:08 GMT
4497 Server: CouchDB (Erlang OTP)
4498 {
4500 "committed_update_seq": 61772,
4501 "compact_running": false,
4502 "db_name": "source",
4503 "disk_format_version": 6,
4504 "doc_count": 41961,
4505 "doc_del_count": 3807,
4506 "instance_start_time": "0",
4507 "purge_seq": 0,
4508 "sizes": {
4509 "active": 70781613961,
4510 "disk": 79132913799,
4511 "external": 72345632950
4512 },
4513 "update_seq": 61772
4514 }
4515 Get Target Information
4517 Request:
4518 GET /target/ HTTP/1.1
4520 Accept: application/json
4521 Host: localhost:5984
4522 User-Agent: CouchDB
4523 Response:
4525 HTTP/1.1 200 OK
4527 Content-Length: 363
4528 Content-Type: application/json
4529 Date: Tue, 08 Oct 2013 12:37:01 GMT
4530 Server: CouchDB (Erlang/OTP)
4531 {
4533 "compact_running": false,
4534 "db_name": "target",
4535 "disk_format_version": 5,
4536 "doc_count": 1832,
4537 "doc_del_count": 1,
4538 "instance_start_time": "0",
4539 "purge_seq": 0,
4540 "sizes": {
4541 "active": 50829452,
4542 "disk": 77001455,
4543 "external": 60326450
4544 },
4545 "update_seq": "1841-g1AAAADveJzLYWBgYMlgTmGQT0lKzi9KdUhJMtbLSs1LLUst0k"
4546 }
4547 Find Common Ancestry
4549 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4550 ' Get Peers Information: '
4551 ' '
4552 ' +-------------------------------------------+ '
4553 ' | Get Target Information | '
4554 ' +-------------------------------------------+ '
4555 ' | '
4556 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4557 |
4558 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4559 ' Find Common Ancestry: v '
4560 ' +-------------------------------------------+ '
4561 ' | Generate Replication ID | '
4562 ' +-------------------------------------------+ '
4563 ' | '
4564 ' | '
4565 ' v '
4566 ' +-------------------------------------------+ '
4567 ' | Get Replication Log from Source | '
4568 ' +-------------------------------------------+ '
4569 ' | GET /source/_local/replication-id | '
4570 ' +-------------------------------------------+ '
4571 ' | '
4572 ' | 200 OK '
4573 ' | 404 Not Found '
4574 ' v '
4575 ' +-------------------------------------------+ '
4576 ' | Get Replication Log from Target | '
4577 ' +-------------------------------------------+ '
4578 ' | GET /target/_local/replication-id | '
4579 ' +-------------------------------------------+ '
4580 ' | '
4581 ' | 200 OK '
4582 ' | 404 Not Found '
4583 ' v '
4584 ' +-------------------------------------------+ '
4585 ' | Compare Replication Logs | '
4586 ' +-------------------------------------------+ '
4587 ' | '
4588 ' | Use latest common sequence as start point '
4589 ' | '
4590 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4591 |
4592 |
4593 + - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - - - - - - - +
4594 ' Locate Changed Documents: | '
4595 ' | '
4596 ' v '
4597 ' +-------------------------------------------+ '
4598 ' | Listen Source Changes Feed | '
4599 ' +-------------------------------------------+ '
4600 ' '
4601 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4602 Generate Replication ID
4604 Before Replication is started, the Replicator MUST generate a Replica‐
4605 tion ID. This value is used to track Replication History, resume and
4606 continue previously interrupted Replication process.
4607 The Replication ID generation algorithm is implementation specific.
4609 Whatever algorithm is used it MUST uniquely identify the Replication
4610 process. CouchDB’s Replicator, for example, uses the following factors
4611 in generating a Replication ID:
4612 · Persistent Peer UUID value. For CouchDB, the local Server UUID is
4614 used
4615 · Source and Target URI and if Source or Target are local or remote
4617 Databases
4618 · If Target needed to be created
4620 · If Replication is Continuous
4622 · Any custom headers
4624 · Filter function code if used
4626 · Changes Feed query parameters, if any
4628 NOTE:
4630 See couch_replicator_ids.erl for an example of a Replication ID gen‐
4631 eration implementation.
4632 Retrieve Replication Logs from Source and Target
4634 Once the Replication ID has been generated, the Replicator SHOULD
4635 retrieve the Replication Log from both Source and Target using GET
4636 /{db}/_local/{docid}:
4637 Request:
4638 GET /source/_local/b3e44b920ee2951cb2e123b63044427a HTTP/1.1
4640 Accept: application/json
4641 Host: localhost:5984
4642 User-Agent: CouchDB
4643 Response:
4645 HTTP/1.1 200 OK
4647 Cache-Control: must-revalidate
4648 Content-Length: 1019
4649 Content-Type: application/json
4650 Date: Thu, 10 Oct 2013 06:18:56 GMT
4651 ETag: "0-8"
4652 Server: CouchDB (Erlang OTP)
4653 {
4655 "_id": "_local/b3e44b920ee2951cb2e123b63044427a",
4656 "_rev": "0-8",
4657 "history": [
4658 {
4659 "doc_write_failures": 0,
4660 "docs_read": 2,
4661 "docs_written": 2,
4662 "end_last_seq": 5,
4663 "end_time": "Thu, 10 Oct 2013 05:56:38 GMT",
4664 "missing_checked": 2,
4665 "missing_found": 2,
4666 "recorded_seq": 5,
4667 "session_id": "d5a34cbbdafa70e0db5cb57d02a6b955",
4668 "start_last_seq": 3,
4669 "start_time": "Thu, 10 Oct 2013 05:56:38 GMT"
4670 },
4671 {
4672 "doc_write_failures": 0,
4673 "docs_read": 1,
4674 "docs_written": 1,
4675 "end_last_seq": 3,
4676 "end_time": "Thu, 10 Oct 2013 05:56:12 GMT",
4677 "missing_checked": 1,
4678 "missing_found": 1,
4679 "recorded_seq": 3,
4680 "session_id": "11a79cdae1719c362e9857cd1ddff09d",
4681 "start_last_seq": 2,
4682 "start_time": "Thu, 10 Oct 2013 05:56:12 GMT"
4683 },
4684 {
4685 "doc_write_failures": 0,
4686 "docs_read": 2,
4687 "docs_written": 2,
4688 "end_last_seq": 2,
4689 "end_time": "Thu, 10 Oct 2013 05:56:04 GMT",
4690 "missing_checked": 2,
4691 "missing_found": 2,
4692 "recorded_seq": 2,
4693 "session_id": "77cdf93cde05f15fcb710f320c37c155",
4694 "start_last_seq": 0,
4695 "start_time": "Thu, 10 Oct 2013 05:56:04 GMT"
4696 }
4697 ],
4698 "replication_id_version": 3,
4699 "session_id": "d5a34cbbdafa70e0db5cb57d02a6b955",
4700 "source_last_seq": 5
4701 }
4702 The Replication Log SHOULD contain the following fields:
4704 · history (array of object): Replication history. Required
4706 · doc_write_failures (number): Number of failed writes
4708 · docs_read (number): Number of read documents
4710 · docs_written (number): Number of written documents
4712 · end_last_seq (number): Last processed Update Sequence ID
4714 · end_time (string): Replication completion timestamp in RFC 5322
4716 format
4717 · missing_checked (number): Number of checked revisions on Source
4719 · missing_found (number): Number of missing revisions found on Target
4721 · recorded_seq (number): Recorded intermediate Checkpoint. Required
4723 · session_id (string): Unique session ID. Commonly, a random UUID
4725 value is used. Required
4726 · start_last_seq (number): Start update Sequence ID
4728 · start_time (string): Replication start timestamp in RFC 5322 format
4730 · replication_id_version (number): Replication protocol version.
4732 Defines Replication ID calculation algorithm, HTTP API calls and the
4733 others routines. Required
4734 · session_id (string): Unique ID of the last session. Shortcut to the
4736 session_id field of the latest history object. Required
4737 · source_last_seq (number): Last processed Checkpoint. Shortcut to the
4739 recorded_seq field of the latest history object. Required
4740 This request MAY fall with a 404 Not Found response:
4742 Request:
4743 GET /source/_local/b6cef528f67aa1a8a014dd1144b10e09 HTTP/1.1
4745 Accept: application/json
4746 Host: localhost:5984
4747 User-Agent: CouchDB
4748 Response:
4750 HTTP/1.1 404 Object Not Found
4752 Cache-Control: must-revalidate
4753 Content-Length: 41
4754 Content-Type: application/json
4755 Date: Tue, 08 Oct 2013 13:31:10 GMT
4756 Server: CouchDB (Erlang OTP)
4757 {
4759 "error": "not_found",
4760 "reason": "missing"
4761 }
4762 That’s OK. This means that there is no information about the current
4764 Replication so it must not have been run previously and as such the
4765 Replicator MUST run a Full Replication.
4766 Compare Replication Logs
4768 If the Replication Logs are successfully retrieved from both Source and
4769 Target then the Replicator MUST determine their common ancestry by fol‐
4770 lowing the next algorithm:
4771 · Compare session_id values for the chronological last session - if
4773 they match both Source and Target have a common Replication history
4774 and it seems to be valid. Use source_last_seq value for the startup
4775 Checkpoint
4776 · In case of mismatch, iterate over the history collection to search
4778 for the latest (chronologically) common session_id for Source and
4779 Target. Use value of recorded_seq field as startup Checkpoint
4780 If Source and Target has no common ancestry, the Replicator MUST run
4782 Full Replication.
4783 Locate Changed Documents
4785 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4786 ' Find Common Ancestry: '
4787 ' '
4788 ' +------------------------------+ '
4789 ' | Compare Replication Logs | '
4790 ' +------------------------------+ '
4791 ' | '
4792 ' | '
4793 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4794 |
4795 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4796 ' Locate Changed Documents: | '
4797 ' | '
4798 ' | '
4799 ' v '
4800 ' +-------------------------------+ '
4801 ' +------> | Listen to Changes Feed | -----+ '
4802 ' | +-------------------------------+ | '
4803 ' | | GET /source/_changes | | '
4804 ' | | POST /source/_changes | | '
4805 ' | +-------------------------------+ | '
4806 ' | | | '
4807 ' | | | '
4808 ' | There are new changes | | No more changes '
4809 ' | | | '
4810 ' | v v '
4811 ' | +-------------------------------+ +-----------------------+ '
4812 ' | | Read Batch of Changes | | Replication Completed | '
4813 ' | +-------------------------------+ +-----------------------+ '
4814 ' | | '
4815 ' | No | '
4816 ' | v '
4817 ' | +-------------------------------+ '
4818 ' | | Compare Documents Revisions | '
4819 ' | +-------------------------------+ '
4820 ' | | POST /target/_revs_diff | '
4821 ' | +-------------------------------+ '
4822 ' | | '
4823 ' | 200 OK | '
4824 ' | v '
4825 ' | +-------------------------------+ '
4826 ' +------- | Any Differences Found? | '
4827 ' +-------------------------------+ '
4828 ' | '
4829 ' Yes | '
4830 ' | '
4831 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4832 |
4833 + - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - - +
4834 ' Replicate Changes: | '
4835 ' v '
4836 ' +-------------------------------+ '
4837 ' | Fetch Next Changed Document | '
4838 ' +-------------------------------+ '
4839 ' '
4840 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
4841 Listen to Changes Feed
4843 When the start up Checkpoint has been defined, the Replicator SHOULD
4844 read the Source’s Changes Feed by using a GET /{db}/_changes request.
4845 This request MUST be made with the following query parameters:
4846 · feed parameter defines the Changes Feed response style: for Continu‐
4848 ous Replication the continuous value SHOULD be used, otherwise - nor‐
4849 mal.
4850 · style=all_docs query parameter tells the Source that it MUST include
4852 all Revision leaves for each document’s event in output.
4853 · For Continuous Replication the heartbeat parameter defines the heart‐
4855 beat period in milliseconds. The RECOMMENDED value by default is
4856 10000 (10 seconds).
4857 · If a startup Checkpoint was found during the Replication Logs compar‐
4859 ison, the since query parameter MUST be passed with this value. In
4860 case of Full Replication it MAY be 0 (number zero) or be omitted.
4861 Additionally, the filter query parameter MAY be specified to enable a
4863 filter function on Source side. Other custom parameters MAY also be
4864 provided.
4865 Read Batch of Changes
4867 Reading the whole feed in a single shot may not be an optimal use of
4868 resources. It is RECOMMENDED to process the feed in small chunks. How‐
4869 ever, there is no specific recommendation on chunk size since it is
4870 heavily dependent on available resources: large chunks requires more
4871 memory while they reduce I/O operations and vice versa.
4872 Note, that Changes Feed output format is different for a request with
4874 feed=normal and with feed=continuous query parameter.
4875 Normal Feed:
4877 Request:
4878 GET /source/_changes?feed=normal&style=all_docs&heartbeat=10000 HTTP/1.1
4880 Accept: application/json
4881 Host: localhost:5984
4882 User-Agent: CouchDB
4883 Response:
4885 HTTP/1.1 200 OK
4887 Cache-Control: must-revalidate
4888 Content-Type: application/json
4889 Date: Fri, 09 May 2014 16:20:41 GMT
4890 Server: CouchDB (Erlang OTP)
4891 Transfer-Encoding: chunked
4892 {"results":[
4894 {"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}
4895 {"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]}
4896 {"seq":37,"id":"d3cc62f5","changes":[{"rev":"2-eec2"}],"deleted":true}
4897 {"seq":39,"id":"f13bd08b","changes":[{"rev":"1-b35d"}]}
4898 {"seq":41,"id":"e0a99867","changes":[{"rev":"2-c1c6"}]}
4899 {"seq":42,"id":"a75bdfc5","changes":[{"rev":"1-967a"}]}
4900 {"seq":43,"id":"a5f467a0","changes":[{"rev":"1-5575"}]}
4901 {"seq":45,"id":"470c3004","changes":[{"rev":"11-c292"}]}
4902 {"seq":46,"id":"b1cb8508","changes":[{"rev":"10-ABC"}]}
4903 {"seq":47,"id":"49ec0489","changes":[{"rev":"157-b01f"},{"rev":"123-6f7c"}]}
4904 {"seq":49,"id":"dad10379","changes":[{"rev":"1-9346"},{"rev":"6-5b8a"}]}
4905 {"seq":50,"id":"73464877","changes":[{"rev":"1-9f08"}]}
4906 {"seq":51,"id":"7ae19302","changes":[{"rev":"1-57bf"}]}
4907 {"seq":63,"id":"6a7a6c86","changes":[{"rev":"5-acf6"}],"deleted":true}
4908 {"seq":64,"id":"dfb9850a","changes":[{"rev":"1-102f"}]}
4909 {"seq":65,"id":"c532afa7","changes":[{"rev":"1-6491"}]}
4910 {"seq":66,"id":"af8a9508","changes":[{"rev":"1-3db2"}]}
4911 {"seq":67,"id":"caa3dded","changes":[{"rev":"1-6491"}]}
4912 {"seq":68,"id":"79f3b4e9","changes":[{"rev":"1-102f"}]}
4913 {"seq":69,"id":"1d89d16f","changes":[{"rev":"1-3db2"}]}
4914 {"seq":71,"id":"abae7348","changes":[{"rev":"2-7051"}]}
4915 {"seq":77,"id":"6c25534f","changes":[{"rev":"9-CDE"},{"rev":"3-00e7"},{"rev":"1-ABC"}]}
4916 {"seq":78,"id":"SpaghettiWithMeatballs","changes":[{"rev":"22-5f95"}]}
4917 ],
4918 "last_seq":78}
4919 Continuous Feed:
4921 Request:
4922 GET /source/_changes?feed=continuous&style=all_docs&heartbeat=10000 HTTP/1.1
4924 Accept: application/json
4925 Host: localhost:5984
4926 User-Agent: CouchDB
4927 Response:
4929 HTTP/1.1 200 OK
4931 Cache-Control: must-revalidate
4932 Content-Type: application/json
4933 Date: Fri, 09 May 2014 16:22:22 GMT
4934 Server: CouchDB (Erlang OTP)
4935 Transfer-Encoding: chunked
4936 {"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}
4938 {"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]}
4939 {"seq":37,"id":"d3cc62f5","changes":[{"rev":"2-eec2"}],"deleted":true}
4940 {"seq":39,"id":"f13bd08b","changes":[{"rev":"1-b35d"}]}
4941 {"seq":41,"id":"e0a99867","changes":[{"rev":"2-c1c6"}]}
4942 {"seq":42,"id":"a75bdfc5","changes":[{"rev":"1-967a"}]}
4943 {"seq":43,"id":"a5f467a0","changes":[{"rev":"1-5575"}]}
4944 {"seq":45,"id":"470c3004","changes":[{"rev":"11-c292"}]}
4945 {"seq":46,"id":"b1cb8508","changes":[{"rev":"10-ABC"}]}
4946 {"seq":47,"id":"49ec0489","changes":[{"rev":"157-b01f"},{"rev":"123-6f7c"}]}
4947 {"seq":49,"id":"dad10379","changes":[{"rev":"1-9346"},{"rev":"6-5b8a"}]}
4948 {"seq":50,"id":"73464877","changes":[{"rev":"1-9f08"}]}
4949 {"seq":51,"id":"7ae19302","changes":[{"rev":"1-57bf"}]}
4950 {"seq":63,"id":"6a7a6c86","changes":[{"rev":"5-acf6"}],"deleted":true}
4951 {"seq":64,"id":"dfb9850a","changes":[{"rev":"1-102f"}]}
4952 {"seq":65,"id":"c532afa7","changes":[{"rev":"1-6491"}]}
4953 {"seq":66,"id":"af8a9508","changes":[{"rev":"1-3db2"}]}
4954 {"seq":67,"id":"caa3dded","changes":[{"rev":"1-6491"}]}
4955 {"seq":68,"id":"79f3b4e9","changes":[{"rev":"1-102f"}]}
4956 {"seq":69,"id":"1d89d16f","changes":[{"rev":"1-3db2"}]}
4957 {"seq":71,"id":"abae7348","changes":[{"rev":"2-7051"}]}
4958 {"seq":75,"id":"SpaghettiWithMeatballs","changes":[{"rev":"21-5949"}]}
4959 {"seq":77,"id":"6c255","changes":[{"rev":"9-CDE"},{"rev":"3-00e7"},{"rev":"1-ABC"}]}
4960 {"seq":78,"id":"SpaghettiWithMeatballs","changes":[{"rev":"22-5f95"}]}
4961 For both Changes Feed formats record-per-line style is preserved to
4963 simplify iterative fetching and decoding JSON objects with less memory
4964 footprint.
4965 Calculate Revision Difference
4967 After reading the batch of changes from the Changes Feed, the Replica‐
4968 tor forms a JSON mapping object for Document ID and related leaf Revi‐
4969 sions and sends the result to Target via a POST /{db}/_revs_diff
4970 request:
4971 Request:
4972 POST /target/_revs_diff HTTP/1.1
4974 Accept: application/json
4975 Content-Length: 287
4976 Content-Type: application/json
4977 Host: localhost:5984
4978 User-Agent: CouchDB
4979 {
4981 "baz": [
4982 "2-7051cbe5c8faecd085a3fa619e6e6337"
4983 ],
4984 "foo": [
4985 "3-6a540f3d701ac518d3b9733d673c5484"
4986 ],
4987 "bar": [
4988 "1-d4e501ab47de6b2000fc8a02f84a0c77",
4989 "1-967a00dff5e02add41819138abb3284d"
4990 ]
4991 }
4992 Response:
4994 HTTP/1.1 200 OK
4996 Cache-Control: must-revalidate
4997 Content-Length: 88
4998 Content-Type: application/json
4999 Date: Fri, 25 Oct 2013 14:44:41 GMT
5000 Server: CouchDB (Erlang/OTP)
5001 {
5003 "baz": {
5004 "missing": [
5005 "2-7051cbe5c8faecd085a3fa619e6e6337"
5006 ]
5007 },
5008 "bar": {
5009 "missing": [
5010 "1-d4e501ab47de6b2000fc8a02f84a0c77"
5011 ]
5012 }
5013 }
5014 In the response the Replicator receives a Document ID – Revisions map‐
5016 ping, but only for Revisions that do not exist in Target and are
5017 REQUIRED to be transferred from Source.
5018 If all Revisions in the request match the current state of the Docu‐
5020 ments then the response will contain an empty JSON object:
5021 Request
5022 POST /target/_revs_diff HTTP/1.1
5024 Accept: application/json
5025 Content-Length: 160
5026 Content-Type: application/json
5027 Host: localhost:5984
5028 User-Agent: CouchDB
5029 {
5031 "foo": [
5032 "3-6a540f3d701ac518d3b9733d673c5484"
5033 ],
5034 "bar": [
5035 "1-967a00dff5e02add41819138abb3284d"
5036 ]
5037 }
5038 Response:
5040 HTTP/1.1 200 OK
5042 Cache-Control: must-revalidate
5043 Content-Length: 2
5044 Content-Type: application/json
5045 Date: Fri, 25 Oct 2013 14:45:00 GMT
5046 Server: CouchDB (Erlang/OTP)
5047 {}
5049 Replication Completed
5051 When there are no more changes left to process and no more Documents
5052 left to replicate, the Replicator finishes the Replication process. If
5053 Replication wasn’t Continuous, the Replicator MAY return a response to
5054 client with statistics about the process.
5055 HTTP/1.1 200 OK
5057 Cache-Control: must-revalidate
5058 Content-Length: 414
5059 Content-Type: application/json
5060 Date: Fri, 09 May 2014 15:14:19 GMT
5061 Server: CouchDB (Erlang OTP)
5062 {
5064 "history": [
5065 {
5066 "doc_write_failures": 2,
5067 "docs_read": 2,
5068 "docs_written": 0,
5069 "end_last_seq": 2939,
5070 "end_time": "Fri, 09 May 2014 15:14:19 GMT",
5071 "missing_checked": 1835,
5072 "missing_found": 2,
5073 "recorded_seq": 2939,
5074 "session_id": "05918159f64842f1fe73e9e2157b2112",
5075 "start_last_seq": 0,
5076 "start_time": "Fri, 09 May 2014 15:14:18 GMT"
5077 }
5078 ],
5079 "ok": true,
5080 "replication_id_version": 3,
5081 "session_id": "05918159f64842f1fe73e9e2157b2112",
5082 "source_last_seq": 2939
5083 }
5084 Replicate Changes
5086 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
5087 ' Locate Changed Documents: '
5088 ' '
5089 ' +-------------------------------------+ '
5090 ' | Any Differences Found? | '
5091 ' +-------------------------------------+ '
5092 ' | '
5093 ' | '
5094 ' | '
5095 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5096 |
5097 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5098 ' Replicate Changes: | '
5099 ' v '
5100 ' +-------------------------------------+ '
5101 ' +---------> | Fetch Next Changed Document | <---------------------+ '
5102 ' | +-------------------------------------+ | '
5103 ' | | GET /source/docid | | '
5104 ' | +-------------------------------------+ | '
5105 ' | | | '
5106 ' | | | '
5107 ' | | 201 Created | '
5108 ' | | 200 OK 401 Unauthorized | '
5109 ' | | 403 Forbidden | '
5110 ' | | | '
5111 ' | v | '
5112 ' | +-------------------------------------+ | '
5113 ' | +------ | Document Has Changed Attachments? | | '
5114 ' | | +-------------------------------------+ | '
5115 ' | | | | '
5116 ' | | | | '
5117 ' | | | Yes | '
5118 ' | | | | '
5119 ' | | v | '
5120 ' | | +------------------------+ Yes +---------------------------+ '
5121 ' | | No | Are They Big Enough? | -------> | Update Document on Target | '
5122 ' | | +------------------------+ +---------------------------+ '
5123 ' | | | | PUT /target/docid | '
5124 ' | | | +---------------------------+ '
5125 ' | | | '
5126 ' | | | No '
5127 ' | | | '
5128 ' | | v '
5129 ' | | +-------------------------------------+ '
5130 ' | +-----> | Put Document Into the Stack | '
5131 ' | +-------------------------------------+ '
5132 ' | | '
5133 ' | | '
5134 ' | v '
5135 ' | No +-------------------------------------+ '
5136 ' +---------- | Stack is Full? | '
5137 ' | +-------------------------------------+ '
5138 ' | | '
5139 ' | | Yes '
5140 ' | | '
5141 ' | v '
5142 ' | +-------------------------------------+ '
5143 ' | | Upload Stack of Documents to Target | '
5144 ' | +-------------------------------------+ '
5145 ' | | POST /target/_bulk_docs | '
5146 ' | +-------------------------------------+ '
5147 ' | | '
5148 ' | | 201 Created '
5149 ' | v '
5150 ' | +-------------------------------------+ '
5151 ' | | Ensure in Commit | '
5152 ' | +-------------------------------------+ '
5153 ' | | POST /target/_ensure_full_commit | '
5154 ' | +-------------------------------------+ '
5155 ' | | '
5156 ' | | 201 Created '
5157 ' | v '
5158 ' | +-------------------------------------+ '
5159 ' | | Record Replication Checkpoint | '
5160 ' | +-------------------------------------+ '
5161 ' | | PUT /source/_local/replication-id | '
5162 ' | | PUT /target/_local/replication-id | '
5163 ' | +-------------------------------------+ '
5164 ' | | '
5165 ' | | 201 Created '
5166 ' | v '
5167 ' | No +-------------------------------------+ '
5168 ' +---------- | All Documents from Batch Processed? | '
5169 ' +-------------------------------------+ '
5170 ' | '
5171 ' Yes | '
5172 ' | '
5173 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5174 |
5175 + - - - - - - - - - - - - - - - - - - - - - - - - - | - - - - - - - - - - - - - - +
5176 ' Locate Changed Documents: | '
5177 ' v '
5178 ' +-------------------------------------+ '
5179 ' | Listen to Changes Feed | '
5180 ' +-------------------------------------+ '
5181 ' '
5182 + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
5183 Fetch Changed Documents
5185 At this step the Replicator MUST fetch all Document Leaf Revisions from
5186 Source that are missed at Target. This operation is effective if Repli‐
5187 cation WILL use previously calculated Revision differences since they
5188 define missing Documents and their Revisions.
5189 To fetch the Document the Replicator will make a GET /{db}/{docid}
5191 request with the following query parameters:
5192 · revs=true: Instructs the Source to include the list of all known
5194 revisions into the Document in the _revisions field. This information
5195 is needed to synchronize the Document’s ancestors history between
5196 Source and Target
5197 · The open_revs query parameter contains a JSON array with a list of
5199 Leaf Revisions that are needed to be fetched. If the specified Revi‐
5200 sion exists then the Document MUST be returned for this Revision.
5201 Otherwise, Source MUST return an object with the single field missing
5202 with the missed Revision as the value. In case the Document contains
5203 attachments, Source MUST return information only for those ones that
5204 had been changed (added or updated) since the specified Revision val‐
5205 ues. If an attachment was deleted, the Document MUST NOT have stub
5206 information for it
5207 · latest=true: Ensures, that Source will return the latest Document
5209 Revision regardless of which one was specified in the open_revs query
5210 parameter. This parameter solves a race condition problem where the
5211 requested Document may be changed in between this step and handling
5212 related events on the Changes Feed
5213 In the response Source SHOULD return multipart/mixed or respond instead
5215 with application/json unless the Accept header specifies a different
5216 mime type. The multipart/mixed content type allows handling the
5217 response data as a stream, since there could be multiple documents (one
5218 per each Leaf Revision) plus several attachments. These attachments are
5219 mostly binary and JSON has no way to handle such data except as base64
5220 encoded strings which are very ineffective for transfer and processing
5221 operations.
5222 With a multipart/mixed response the Replicator handles multiple Docu‐
5224 ment Leaf Revisions and their attachments one by one as raw data with‐
5225 out any additional encoding applied. There is also one agreement to
5226 make data processing more effective: the Document ALWAYS goes before
5227 its attachments, so the Replicator has no need to process all the data
5228 to map related Documents-Attachments and may handle it as stream with
5229 lesser memory footprint.
5230 Request:
5231 GET /source/SpaghettiWithMeatballs?revs=true&open_revs=[%225-00ecbbc%22,%221-917fa23%22,%223-6bcedf1%22]&latest=true HTTP/1.1
5233 Accept: multipart/mixed
5234 Host: localhost:5984
5235 User-Agent: CouchDB
5236 Response:
5238 HTTP/1.1 200 OK
5240 Content-Type: multipart/mixed; boundary="7b1596fc4940bc1be725ad67f11ec1c4"
5241 Date: Thu, 07 Nov 2013 15:10:16 GMT
5242 Server: CouchDB (Erlang OTP)
5243 Transfer-Encoding: chunked
5244 --7b1596fc4940bc1be725ad67f11ec1c4
5246 Content-Type: application/json
5247 {
5249 "_id": "SpaghettiWithMeatballs",
5250 "_rev": "1-917fa23",
5251 "_revisions": {
5252 "ids": [
5253 "917fa23"
5254 ],
5255 "start": 1
5256 },
5257 "description": "An Italian-American delicious dish",
5258 "ingredients": [
5259 "spaghetti",
5260 "tomato sauce",
5261 "meatballs"
5262 ],
5263 "name": "Spaghetti with meatballs"
5264 }
5265 --7b1596fc4940bc1be725ad67f11ec1c4
5266 Content-Type: multipart/related; boundary="a81a77b0ca68389dda3243a43ca946f2"
5267 --a81a77b0ca68389dda3243a43ca946f2
5269 Content-Type: application/json
5270 {
5272 "_attachments": {
5273 "recipe.txt": {
5274 "content_type": "text/plain",
5275 "digest": "md5-R5CrCb6fX10Y46AqtNn0oQ==",
5276 "follows": true,
5277 "length": 87,
5278 "revpos": 7
5279 }
5280 },
5281 "_id": "SpaghettiWithMeatballs",
5282 "_rev": "7-474f12e",
5283 "_revisions": {
5284 "ids": [
5285 "474f12e",
5286 "5949cfc",
5287 "00ecbbc",
5288 "fc997b6",
5289 "3552c87",
5290 "404838b",
5291 "5defd9d",
5292 "dc1e4be"
5293 ],
5294 "start": 7
5295 },
5296 "description": "An Italian-American delicious dish",
5297 "ingredients": [
5298 "spaghetti",
5299 "tomato sauce",
5300 "meatballs",
5301 "love"
5302 ],
5303 "name": "Spaghetti with meatballs"
5304 }
5305 --a81a77b0ca68389dda3243a43ca946f2
5306 Content-Disposition: attachment; filename="recipe.txt"
5307 Content-Type: text/plain
5308 Content-Length: 87
5309 1. Cook spaghetti
5311 2. Cook meetballs
5312 3. Mix them
5313 4. Add tomato sauce
5314 5. ...
5315 6. PROFIT!
5316 --a81a77b0ca68389dda3243a43ca946f2--
5318 --7b1596fc4940bc1be725ad67f11ec1c4
5319 Content-Type: application/json; error="true"
5320 {"missing":"3-6bcedf1"}
5322 --7b1596fc4940bc1be725ad67f11ec1c4--
5323 After receiving the response, the Replicator puts all the received data
5325 into a local stack for further bulk upload to utilize network bandwidth
5326 effectively. The local stack size could be limited by number of Docu‐
5327 ments or bytes of handled JSON data. When the stack is full the Repli‐
5328 cator uploads all the handled Document in bulk mode to the Target.
5329 While bulk operations are highly RECOMMENDED to be used, in certain
5330 cases the Replicator MAY upload Documents to Target one by one.
5331 NOTE:
5333 Alternative Replicator implementations MAY use alternative ways to
5334 retrieve Documents from Source. For instance, PouchDB doesn’t use
5335 the Multipart API and fetches only the latest Document Revision with
5336 inline attachments as a single JSON object. While this is still
5337 valid CouchDB HTTP API usage, such solutions MAY require a different
5338 API implementation for non-CouchDB Peers.
5339 Upload Batch of Changed Documents
5341 To upload multiple Documents in a single shot the Replicator sends a
5342 POST /{db}/_bulk_docs request to Target with payload containing a JSON
5343 object with the following mandatory fields:
5344 · docs (array of objects): List of Document objects to update on Tar‐
5346 get. These Documents MUST contain the _revisions field that holds a
5347 list of the full Revision history to let Target create Leaf Revisions
5348 that correctly preserve ancestry
5349 · new_edits (boolean): Special flag that instructs Target to store Doc‐
5351 uments with the specified Revision (field _rev) value as-is without
5352 generating a new revision. Always false
5353 The request also MAY contain X-Couch-Full-Commit that used to control
5355 CouchDB <3.0 behavior when delayed commits were enabled. Other Peers
5356 MAY ignore this header or use it to control similar local feature.
5357 Request:
5358 POST /target/_bulk_docs HTTP/1.1
5360 Accept: application/json
5361 Content-Length: 826
5362 Content-Type:application/json
5363 Host: localhost:5984
5364 User-Agent: CouchDB
5365 X-Couch-Full-Commit: false
5366 {
5368 "docs": [
5369 {
5370 "_id": "SpaghettiWithMeatballs",
5371 "_rev": "1-917fa2381192822767f010b95b45325b",
5372 "_revisions": {
5373 "ids": [
5374 "917fa2381192822767f010b95b45325b"
5375 ],
5376 "start": 1
5377 },
5378 "description": "An Italian-American delicious dish",
5379 "ingredients": [
5380 "spaghetti",
5381 "tomato sauce",
5382 "meatballs"
5383 ],
5384 "name": "Spaghetti with meatballs"
5385 },
5386 {
5387 "_id": "LambStew",
5388 "_rev": "1-34c318924a8f327223eed702ddfdc66d",
5389 "_revisions": {
5390 "ids": [
5391 "34c318924a8f327223eed702ddfdc66d"
5392 ],
5393 "start": 1
5394 },
5395 "servings": 6,
5396 "subtitle": "Delicious with scone topping",
5397 "title": "Lamb Stew"
5398 },
5399 {
5400 "_id": "FishStew",
5401 "_rev": "1-9c65296036141e575d32ba9c034dd3ee",
5402 "_revisions": {
5403 "ids": [
5404 "9c65296036141e575d32ba9c034dd3ee"
5405 ],
5406 "start": 1
5407 },
5408 "servings": 4,
5409 "subtitle": "Delicious with fresh bread",
5410 "title": "Fish Stew"
5411 }
5412 ],
5413 "new_edits": false
5414 }
5415 In its response Target MUST return a JSON array with a list of Document
5417 update statuses. If the Document has been stored successfully, the list
5418 item MUST contain the field ok with true value. Otherwise it MUST con‐
5419 tain error and reason fields with error type and a human-friendly rea‐
5420 son description.
5421 Document updating failure isn’t fatal as Target MAY reject the update
5423 for its own reasons. It’s RECOMMENDED to use error type forbidden for
5424 rejections, but other error types can also be used (like invalid field
5425 name etc.). The Replicator SHOULD NOT retry uploading rejected docu‐
5426 ments unless there are good reasons for doing so (e.g. there is special
5427 error type for that).
5428 Note that while a update may fail for one Document in the response,
5430 Target can still return a 201 Created response. Same will be true if
5431 all updates fail for all uploaded Documents.
5432 Response:
5433 HTTP/1.1 201 Created
5435 Cache-Control: must-revalidate
5436 Content-Length: 246
5437 Content-Type: application/json
5438 Date: Sun, 10 Nov 2013 19:02:26 GMT
5439 Server: CouchDB (Erlang/OTP)
5440 [
5442 {
5443 "ok": true,
5444 "id": "SpaghettiWithMeatballs",
5445 "rev":" 1-917fa2381192822767f010b95b45325b"
5446 },
5447 {
5448 "ok": true,
5449 "id": "FishStew",
5450 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
5451 },
5452 {
5453 "error": "forbidden",
5454 "id": "LambStew",
5455 "reason": "sorry",
5456 "rev": "1-34c318924a8f327223eed702ddfdc66d"
5457 }
5458 ]
5459 Upload Document with Attachments
5461 There is a special optimization case when then Replicator WILL NOT use
5462 bulk upload of changed Documents. This case is applied when Documents
5463 contain a lot of attached files or the files are too big to be effi‐
5464 ciently encoded with Base64.
5465 For this case the Replicator issues a /{db}/{docid}?new_edits=false
5467 request with multipart/related content type. Such a request allows one
5468 to easily stream the Document and all its attachments one by one with‐
5469 out any serialization overhead.
5470 Request:
5471 PUT /target/SpaghettiWithMeatballs?new_edits=false HTTP/1.1
5473 Accept: application/json
5474 Content-Length: 1030
5475 Content-Type: multipart/related; boundary="864d690aeb91f25d469dec6851fb57f2"
5476 Host: localhost:5984
5477 User-Agent: CouchDB
5478 --2fa48cba80d0cdba7829931fe8acce9d
5480 Content-Type: application/json
5481 {
5483 "_attachments": {
5484 "recipe.txt": {
5485 "content_type": "text/plain",
5486 "digest": "md5-R5CrCb6fX10Y46AqtNn0oQ==",
5487 "follows": true,
5488 "length": 87,
5489 "revpos": 7
5490 }
5491 },
5492 "_id": "SpaghettiWithMeatballs",
5493 "_rev": "7-474f12eb068c717243487a9505f6123b",
5494 "_revisions": {
5495 "ids": [
5496 "474f12eb068c717243487a9505f6123b",
5497 "5949cfcd437e3ee22d2d98a26d1a83bf",
5498 "00ecbbc54e2a171156ec345b77dfdf59",
5499 "fc997b62794a6268f2636a4a176efcd6",
5500 "3552c87351aadc1e4bea2461a1e8113a",
5501 "404838bc2862ce76c6ebed046f9eb542",
5502 "5defd9d813628cea6e98196eb0ee8594"
5503 ],
5504 "start": 7
5505 },
5506 "description": "An Italian-American delicious dish",
5507 "ingredients": [
5508 "spaghetti",
5509 "tomato sauce",
5510 "meatballs",
5511 "love"
5512 ],
5513 "name": "Spaghetti with meatballs"
5514 }
5515 --2fa48cba80d0cdba7829931fe8acce9d
5516 Content-Disposition: attachment; filename="recipe.txt"
5517 Content-Type: text/plain
5518 Content-Length: 87
5519 1. Cook spaghetti
5521 2. Cook meetballs
5522 3. Mix them
5523 4. Add tomato sauce
5524 5. ...
5525 6. PROFIT!
5526 --2fa48cba80d0cdba7829931fe8acce9d--
5528 Response:
5530 HTTP/1.1 201 Created
5532 Cache-Control: must-revalidate
5533 Content-Length: 105
5534 Content-Type: application/json
5535 Date: Fri, 08 Nov 2013 16:35:27 GMT
5536 Server: CouchDB (Erlang/OTP)
5537 {
5539 "ok": true,
5540 "id": "SpaghettiWithMeatballs",
5541 "rev": "7-474f12eb068c717243487a9505f6123b"
5542 }
5543 Unlike bulk updating via POST /{db}/_bulk_docs endpoint, the response
5545 MAY come with a different status code. For instance, in the case when
5546 the Document is rejected, Target SHOULD respond with a 403 Forbidden:
5547 Response:
5548 HTTP/1.1 403 Forbidden
5550 Cache-Control: must-revalidate
5551 Content-Length: 39
5552 Content-Type: application/json
5553 Date: Fri, 08 Nov 2013 16:35:27 GMT
5554 Server: CouchDB (Erlang/OTP)
5555 {
5557 "error": "forbidden",
5558 "reason": "sorry"
5559 }
5560 Replicator SHOULD NOT retry requests in case of a 401 Unauthorized, 403
5562 Forbidden, 409 Conflict or 412 Precondition Failed since repeating the
5563 request couldn’t solve the issue with user credentials or uploaded
5564 data.
5565 Ensure In Commit
5567 Once a batch of changes has been successfully uploaded to Target, the
5568 Replicator issues a POST /{db}/_ensure_full_commit request to ensure
5569 that every transferred bit is laid down on disk or other persistent
5570 storage place. Target MUST return 201 Created response with a JSON
5571 object containing the following mandatory fields:
5572 · instance_start_time (string): Timestamp of when the database was
5574 opened, expressed in microseconds since the epoch
5575 · ok (boolean): Operation status. Constantly true
5577 Request:
5579 POST /target/_ensure_full_commit HTTP/1.1
5581 Accept: application/json
5582 Content-Type: application/json
5583 Host: localhost:5984
5584 Response:
5586 HTTP/1.1 201 Created
5588 Cache-Control: must-revalidate
5589 Content-Length: 53
5590 Content-Type: application/json
5591 Date: Web, 06 Nov 2013 18:20:43 GMT
5592 Server: CouchDB (Erlang/OTP)
5593 {
5595 "instance_start_time": "0",
5596 "ok": true
5597 }
5598 Record Replication Checkpoint
5600 Since batches of changes were uploaded and committed successfully, the
5601 Replicator updates the Replication Log both on Source and Target
5602 recording the current Replication state. This operation is REQUIRED so
5603 that in the case of Replication failure the replication can resume from
5604 last point of success, not from the very beginning.
5605 Replicator updates Replication Log on Source:
5607 Request:
5608 PUT /source/_local/afa899a9e59589c3d4ce5668e3218aef HTTP/1.1
5610 Accept: application/json
5611 Content-Length: 591
5612 Content-Type: application/json
5613 Host: localhost:5984
5614 User-Agent: CouchDB
5615 {
5617 "_id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5618 "_rev": "0-1",
5619 "_revisions": {
5620 "ids": [
5621 "31f36e40158e717fbe9842e227b389df"
5622 ],
5623 "start": 1
5624 },
5625 "history": [
5626 {
5627 "doc_write_failures": 0,
5628 "docs_read": 6,
5629 "docs_written": 6,
5630 "end_last_seq": 26,
5631 "end_time": "Thu, 07 Nov 2013 09:42:17 GMT",
5632 "missing_checked": 6,
5633 "missing_found": 6,
5634 "recorded_seq": 26,
5635 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5636 "start_last_seq": 0,
5637 "start_time": "Thu, 07 Nov 2013 09:41:43 GMT"
5638 }
5639 ],
5640 "replication_id_version": 3,
5641 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5642 "source_last_seq": 26
5643 }
5644 Response:
5646 HTTP/1.1 201 Created
5648 Cache-Control: must-revalidate
5649 Content-Length: 75
5650 Content-Type: application/json
5651 Date: Thu, 07 Nov 2013 09:42:17 GMT
5652 Server: CouchDB (Erlang/OTP)
5653 {
5655 "id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5656 "ok": true,
5657 "rev": "0-2"
5658 }
5659 …and on Target too:
5661 Request:
5662 PUT /target/_local/afa899a9e59589c3d4ce5668e3218aef HTTP/1.1
5664 Accept: application/json
5665 Content-Length: 591
5666 Content-Type: application/json
5667 Host: localhost:5984
5668 User-Agent: CouchDB
5669 {
5671 "_id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5672 "_rev": "1-31f36e40158e717fbe9842e227b389df",
5673 "_revisions": {
5674 "ids": [
5675 "31f36e40158e717fbe9842e227b389df"
5676 ],
5677 "start": 1
5678 },
5679 "history": [
5680 {
5681 "doc_write_failures": 0,
5682 "docs_read": 6,
5683 "docs_written": 6,
5684 "end_last_seq": 26,
5685 "end_time": "Thu, 07 Nov 2013 09:42:17 GMT",
5686 "missing_checked": 6,
5687 "missing_found": 6,
5688 "recorded_seq": 26,
5689 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5690 "start_last_seq": 0,
5691 "start_time": "Thu, 07 Nov 2013 09:41:43 GMT"
5692 }
5693 ],
5694 "replication_id_version": 3,
5695 "session_id": "04bf15bf1d9fa8ac1abc67d0c3e04f07",
5696 "source_last_seq": 26
5697 }
5698 Response:
5700 HTTP/1.1 201 Created
5702 Cache-Control: must-revalidate
5703 Content-Length: 106
5704 Content-Type: application/json
5705 Date: Thu, 07 Nov 2013 09:42:17 GMT
5706 Server: CouchDB (Erlang/OTP)
5707 {
5709 "id": "_local/afa899a9e59589c3d4ce5668e3218aef",
5710 "ok": true,
5711 "rev": "2-9b5d1e36bed6ae08611466e30af1259a"
5712 }
5713 Continue Reading Changes
5715 Once a batch of changes had been processed and transferred to Target
5716 successfully, the Replicator can continue to listen to the Changes Feed
5717 for new changes. If there are no new changes to process the Replication
5718 is considered to be done.
5719 For Continuous Replication, the Replicator MUST continue to wait for
5721 new changes from Source.
5722 Protocol Robustness
5724 Since the CouchDB Replication Protocol works on top of HTTP, which is
5725 based on TCP/IP, the Replicator SHOULD expect to be working within an
5726 unstable environment with delays, losses and other bad surprises that
5727 might eventually occur. The Replicator SHOULD NOT count every HTTP
5728 request failure as a fatal error. It SHOULD be smart enough to detect
5729 timeouts, repeat failed requests, be ready to process incomplete or
5730 malformed data and so on. Data must flow - that’s the rule.
5731 Error Responses
5733 In case something goes wrong the Peer MUST respond with a JSON object
5734 with the following REQUIRED fields:
5735 · error (string): Error type for programs and developers
5737 · reason (string): Error description for humans
5739 Bad Request
5741 If a request contains malformed data (like invalid JSON) the Peer MUST
5742 respond with a HTTP 400 Bad Request and bad_request as error type:
5743 {
5745 "error": "bad_request",
5746 "reason": "invalid json"
5747 }
5748 Unauthorized
5750 If a Peer REQUIRES credentials be included with the request and the
5751 request does not contain acceptable credentials then the Peer MUST
5752 respond with the HTTP 401 Unauthorized and unauthorized as error type:
5753 {
5755 "error": "unauthorized",
5756 "reason": "Name or password is incorrect"
5757 }
5758 Forbidden
5760 If a Peer receives valid user credentials, but the requester does not
5761 have sufficient permissions to perform the operation then the Peer MUST
5762 respond with a HTTP 403 Forbidden and forbidden as error type:
5763 {
5765 "error": "forbidden",
5766 "reason": "You may only update your own user document."
5767 }
5768 Resource Not Found
5770 If the requested resource, Database or Document wasn’t found on a Peer,
5771 the Peer MUST respond with a HTTP 404 Not Found and not_found as error
5772 type:
5773 {
5775 "error": "not_found",
5776 "reason": "database \"target\" does not exists"
5777 }
5778 Method Not Allowed
5780 If an unsupported method was used then the Peer MUST respond with a
5781 HTTP 405 Method Not Allowed and method_not_allowed as error type:
5782 {
5784 "error": "method_not_allowed",
5785 "reason": "Only GET, PUT, DELETE allowed"
5786 }
5787 Resource Conflict
5789 A resource conflict error occurs when there are concurrent updates of
5790 the same resource by multiple clients. In this case the Peer MUST
5791 respond with a HTTP 409 Conflict and conflict as error type:
5792 {
5794 "error": "conflict",
5795 "reason": "document update conflict"
5796 }
5797 Precondition Failed
5799 The HTTP 412 Precondition Failed response may be sent in case of an
5800 attempt to create a Database (error type db_exists) that already exists
5801 or some attachment information is missing (error type missing_stub).
5802 There is no explicit error type restrictions, but it is RECOMMEND to
5803 use error types that are previously mentioned:
5804 {
5806 "error": "db_exists",
5807 "reason": "database \"target\" exists"
5808 }
5809 Server Error
5811 Raised in case an error is fatal and the Replicator cannot do anything
5812 to continue Replication. In this case the Replicator MUST return a HTTP
5813 500 Internal Server Error response with an error description (no
5814 restrictions on error type applied):
5815 {
5817 "error": "worker_died",
5818 "reason": "kaboom!"
5819 }
5820 Optimisations
5822 There are RECOMMENDED approaches to optimize the Replication process:
5823 · Keep the number of HTTP requests at a reasonable minimum
5825 · Try to work with a connection pool and make parallel/multiple
5827 requests whenever possible
5828 · Don’t close sockets after each request: respect the keep-alive option
5830 · Use continuous sessions (cookies, etc.) to reduce authentication
5832 overhead
5833 · Try to use bulk requests for every operations with Documents
5835 · Find out optimal batch size for Changes feed processing
5837 · Preserve Replication Logs and resume Replication from the last Check‐
5839 point whenever possible
5840 · Optimize filter functions: let them run as fast as possible
5842 · Get ready for surprises: networks are very unstable environments
5844 API Reference
5846 Common Methods
5847 · HEAD /{db} – Check Database existence
5848 · GET /{db} – Retrieve Database information
5850 · GET /{db}/_local/{docid} – Read the last Checkpoint
5852 · PUT /{db}/_local/{docid} – Save a new Checkpoint
5854 For Target
5856 · PUT /{db} – Create Target if it not exists and the option was pro‐
5857 vided
5858 · POST /{db}/_revs_diff – Locate Revisions that are not known to Target
5860 · POST /{db}/_bulk_docs – Upload Revisions to Target
5862 · PUT /{db}/{docid} – Upload a single Document with attachments to Tar‐
5864 get
5865 · POST /{db}/_ensure_full_commit – Ensure that all changes are stored
5867 on disk
5868 For Source
5870 · GET /{db}/_changes – Fetch changes since the last pull of Source
5871 · POST /{db}/_changes – Fetch changes for specified Document IDs since
5873 the last pull of Source
5874 · GET /{db}/{docid} – Retrieve a single Document from Source with
5876 attachments
5877 Reference
5879 · Refuge RCouch wiki
5880 · CouchBase Lite IOS wiki
5882
5884 CouchDB supports special documents within databases known as “design
5885 documents”. These documents, mostly driven by JavaScript you write, are
5886 used to build indexes, validate document updates, format query results,
5887 and filter replications.
5888 Design Documents
5890 In this section we’ll show how to write design documents, using the
5891 built-in JavaScript Query Server.
5892 But before we start to write our first document, let’s take a look at
5894 the list of common objects that will be used during our code journey -
5895 we’ll be using them extensively within each function:
5896 · Database information object
5898 · Request object
5900 · Response object
5902 · UserCtx object
5904 · Database Security object
5906 · Guide to JavaScript Query Server
5908 Creation
5910 Design documents are denoted by an id field with the format
5911 _design/{name}.
5912 Example:
5914 {
5916 "id": "_design/example",
5917 }
5918 View Functions
5920 Views are the primary tool used for querying and reporting on CouchDB
5921 databases.
5922 Map Functions
5924 mapfun(doc)
5925 Arguments
5927 · doc – The document that is being processed
5929 Map functions accept a single document as the argument and (optionally)
5931 emit() key/value pairs that are stored in a view.
5932 function (doc) {
5934 if (doc.type === 'post' && doc.tags && Array.isArray(doc.tags)) {
5935 doc.tags.forEach(function (tag) {
5936 emit(tag.toLowerCase(), 1);
5937 });
5938 }
5939 }
5940 In this example a key/value pair is emitted for each value in the tags
5942 array of a document with a type of “post”. Note that emit() may be
5943 called many times for a single document, so the same document may be
5944 available by several different keys.
5945 Also keep in mind that each document is sealed to prevent the situation
5947 where one map function changes document state and another receives a
5948 modified version.
5949 For efficiency reasons, documents are passed to a group of map func‐
5951 tions - each document is processed by a group of map functions from all
5952 views of the related design document. This means that if you trigger an
5953 index update for one view in the design document, all others will get
5954 updated too.
5955 Since version 1.1.0, map supports CommonJS modules and the require()
5957 function.
5958 Reduce and Rereduce Functions
5960 redfun(keys, values[, rereduce])
5961 Arguments
5963 · keys – Array of pairs of docid-key for related map
5965 function results. Always null if rereduce is running
5966 (has true value).
5967 · values – Array of map function result values.
5969 · rereduce – Boolean flag to indicate a rereduce run.
5971 Returns
5973 Reduces values
5974 Reduce functions take two required arguments of keys and values lists -
5976 the result of the related map function - and an optional third value
5977 which indicates if rereduce mode is active or not. Rereduce is used for
5978 additional reduce values list, so when it is true there is no informa‐
5979 tion about related keys (first argument is null).
5980 Note that if the result of a reduce function is longer than the initial
5982 values list then a Query Server error will be raised. However, this
5983 behavior can be disabled by setting reduce_limit config option to
5984 false:
5985 [query_server_config]
5987 reduce_limit = false
5988 While disabling reduce_limit might be useful for debug proposes, remem‐
5990 ber that the main task of reduce functions is to reduce the mapped
5991 result, not to make it bigger. Generally, your reduce function should
5992 converge rapidly to a single value - which could be an array or similar
5993 object.
5994 Built-in Reduce Functions
5996 Additionally, CouchDB has a set of built-in reduce functions. These are
5997 implemented in Erlang and run inside CouchDB, so they are much faster
5998 than the equivalent JavaScript functions.
5999 _approx_count_distinct
6001 New in version 2.2.
6003 Aproximates the number of distinct keys in a view index using a variant
6006 of the HyperLogLog algorithm. This algorithm enables an efficient, par‐
6007 allelizable computation of cardinality using fixed memory resources.
6008 CouchDB has configured the underlying data structure to have a relative
6009 error of ~2%.
6010 As this reducer ignores the emitted values entirely, an invocation with
6012 group=true will simply return a value of 1 for every distinct key in
6013 the view. In the case of array keys, querying the view with a
6014 group_level specified will return the number of distinct keys that
6015 share the common group prefix in each row. The algorithm is also cog‐
6016 nizant of the startkey and endkey boundaries and will return the number
6017 of distinct keys within the specified key range.
6018 A final note regarding Unicode collation: this reduce function uses the
6020 binary representation of each key in the index directly as input to the
6021 HyperLogLog filter. As such, it will (incorrectly) consider keys that
6022 are not byte identical but that compare equal according to the Unicode
6023 collation rules to be distinct keys, and thus has the potential to
6024 overestimate the cardinality of the key space if a large number of such
6025 keys exist.
6026 _count
6028 Counts the number of values in the index with a given key. This could
6030 be implemented in JavaScript as:
6031 // could be replaced by _count
6033 function(keys, values, rereduce) {
6034 if (rereduce) {
6035 return sum(values);
6036 } else {
6037 return values.length;
6038 }
6039 }
6040 _stats
6042 Computes the following quantities for numeric values associated with
6044 each key: sum, min, max, count, and sumsqr. The behavior of the _stats
6045 function varies depending on the output of the map function. The sim‐
6046 plest case is when the map phase emits a single numeric value for each
6047 key. In this case the _stats function is equivalent to the following
6048 JavaScript:
6049 // could be replaced by _stats
6051 function(keys, values, rereduce) {
6052 if (rereduce) {
6053 return {
6054 'sum': values.reduce(function(a, b) { return a + b.sum }, 0),
6055 'min': values.reduce(function(a, b) { return Math.min(a, b.min) }, Infinity),
6056 'max': values.reduce(function(a, b) { return Math.max(a, b.max) }, -Infinity),
6057 'count': values.reduce(function(a, b) { return a + b.count }, 0),
6058 'sumsqr': values.reduce(function(a, b) { return a + b.sumsqr }, 0)
6059 }
6060 } else {
6061 return {
6062 'sum': sum(values),
6063 'min': Math.min.apply(null, values),
6064 'max': Math.max.apply(null, values),
6065 'count': values.length,
6066 'sumsqr': (function() {
6067 var sumsqr = 0;
6068 values.forEach(function (value) {
6070 sumsqr += value * value;
6071 });
6072 return sumsqr;
6074 })(),
6075 }
6076 }
6077 }
6078 The _stats function will also work with “pre-aggregated” values from a
6080 map phase. A map function that emits an object containing sum, min,
6081 max, count, and sumsqr keys and numeric values for each can use the
6082 _stats function to combine these results with the data from other docu‐
6083 ments. The emitted object may contain other keys (these are ignored by
6084 the reducer), and it is also possible to mix raw numeric values and
6085 pre-aggregated objects in a single view and obtain the correct aggre‐
6086 gated statistics.
6087 Finally, _stats can operate on key-value pairs where each value is an
6089 array comprised of numbers or pre-aggregated objects. In this case
6090 every value emitted from the map function must be an array, and the
6091 arrays must all be the same length, as _stats will compute the statis‐
6092 tical quantities above independently for each element in the array.
6093 Users who want to compute statistics on multiple values from a single
6094 document should either emit each value into the index separately, or
6095 compute the statistics for the set of values using the JavaScript exam‐
6096 ple above and emit a pre-aggregated object.
6097 _sum
6099 In its simplest variation, _sum sums the numeric values associated with
6101 each key, as in the following JavaScript:
6102 // could be replaced by _sum
6104 function(keys, values) {
6105 return sum(values);
6106 }
6107 As with _stats, the _sum function offers a number of extended capabili‐
6109 ties. The _sum function requires that map values be numbers, arrays of
6110 numbers, or objects. When presented with array output from a map func‐
6111 tion, _sum will compute the sum for every element of the array. A bare
6112 numeric value will be treated as an array with a single element, and
6113 arrays with fewer elements will be treated as if they contained zeroes
6114 for every additional element in the longest emitted array. As an exam‐
6115 ple, consider the following map output:
6116 {"total_rows":5, "offset":0, "rows": [
6118 {"id":"id1", "key":"abc", "value": 2},
6119 {"id":"id2", "key":"abc", "value": [3,5,7]},
6120 {"id":"id2", "key":"def", "value": [0,0,0,42]},
6121 {"id":"id2", "key":"ghi", "value": 1},
6122 {"id":"id1", "key":"ghi", "value": 3}
6123 ]}
6124 The _sum for this output without any grouping would be:
6126 {"rows": [
6128 {"key":null, "value": [9,5,7,42]}
6129 ]}
6130 while the grouped output would be
6132 {"rows": [
6134 {"key":"abc", "value": [5,5,7]},
6135 {"key":"def", "value": [0,0,0,42]},
6136 {"key":"ghi", "value": 4
6137 ]}
6138 This is in contrast to the behavior of the _stats function which
6140 requires that all emitted values be arrays of identical length if any
6141 array is emitted.
6142 It is also possible to have _sum recursively descend through an emitted
6144 object and compute the sums for every field in the object. Objects can‐
6145 not be mixed with other data structures. Objects can be arbitrarily
6146 nested, provided that the values for all fields are themselves numbers,
6147 arrays of numbers, or objects.
6148 NOTE:
6150 Why don’t reduce functions support CommonJS modules?
6151 While map functions have limited access to stored modules through
6153 require(), there is no such feature for reduce functions. The rea‐
6154 son lies deep inside the way map and reduce functions are processed
6155 by the Query Server. Let’s take a look at map functions first:
6156 1. CouchDB sends all map functions in a processed design document to
6158 the Query Server.
6159 2. the Query Server handles them one by one, compiles and puts them
6161 onto an internal stack.
6162 3. after all map functions have been processed, CouchDB will send
6164 the remaining documents for indexing, one by one.
6165 4. the Query Server receives the document object and applies it to
6167 every function from the stack. The emitted results are then
6168 joined into a single array and sent back to CouchDB.
6169 Now let’s see how reduce functions are handled:
6171 1. CouchDB sends as a single command the list of available reduce
6173 functions with the result list of key-value pairs that were pre‐
6174 viously returned from the map functions.
6175 2. the Query Server compiles the reduce functions and applies them
6177 to the key-value lists. The reduced result is sent back to
6178 CouchDB.
6179 As you may note, reduce functions are applied in a single shot to
6181 the map results while map functions are applied to documents one by
6182 one. This means that it’s possible for map functions to precompile
6183 CommonJS libraries and use them during the entire view processing,
6184 but for reduce functions they would be compiled again and again for
6185 each view result reduction, which would lead to performance degrada‐
6186 tion.
6187 Show Functions
6189 WARNING:
6190 Show functions are deprecated in CouchDB 3.0, and will be removed in
6191 CouchDB 4.0.
6192 showfun(doc, req)
6194 Arguments
6196 · doc – The document that is being processed; may be
6198 omitted.
6199 · req – Request object.
6201 Returns
6203 Response object
6204 Return type
6206 object or string
6207 Show functions are used to represent documents in various formats, com‐
6209 monly as HTML pages with nice formatting. They can also be used to run
6210 server-side functions without requiring a pre-existing document.
6211 Basic example of show function could be:
6213 function(doc, req){
6215 if (doc) {
6216 return "Hello from " + doc._id + "!";
6217 } else {
6218 return "Hello, world!";
6219 }
6220 }
6221 Also, there is more simple way to return json encoded data:
6223 function(doc, req){
6225 return {
6226 'json': {
6227 'id': doc['_id'],
6228 'rev': doc['_rev']
6229 }
6230 }
6231 }
6232 and even files (this one is CouchDB logo):
6234 function(doc, req){
6236 return {
6237 'headers': {
6238 'Content-Type' : 'image/png',
6239 },
6240 'base64': ''.concat(
6241 'iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAMAAAAoLQ9TAAAAsV',
6242 'BMVEUAAAD////////////////////////5ur3rEBn////////////////wDBL/',
6243 'AADuBAe9EB3IEBz/7+//X1/qBQn2AgP/f3/ilpzsDxfpChDtDhXeCA76AQH/v7',
6244 '/84eLyWV/uc3bJPEf/Dw/uw8bRWmP1h4zxSlD6YGHuQ0f6g4XyQkXvCA36MDH6',
6245 'wMH/z8/yAwX64ODeh47BHiv/Ly/20dLQLTj98PDXWmP/Pz//39/wGyJ7Iy9JAA',
6246 'AADHRSTlMAbw8vf08/bz+Pv19jK/W3AAAAg0lEQVR4Xp3LRQ4DQRBD0QqTm4Y5',
6247 'zMxw/4OleiJlHeUtv2X6RbNO1Uqj9g0RMCuQO0vBIg4vMFeOpCWIWmDOw82fZx',
6248 'vaND1c8OG4vrdOqD8YwgpDYDxRgkSm5rwu0nQVBJuMg++pLXZyr5jnc1BaH4GT',
6249 'LvEliY253nA3pVhQqdPt0f/erJkMGMB8xucAAAAASUVORK5CYII=')
6250 }
6251 }
6252 But what if you need to represent data in different formats via a sin‐
6254 gle function? Functions registerType() and provides() are your the best
6255 friends in that question:
6256 function(doc, req){
6258 provides('json', function(){
6259 return {'json': doc}
6260 });
6261 provides('html', function(){
6262 return '<pre>' + toJSON(doc) + '</pre>'
6263 })
6264 provides('xml', function(){
6265 return {
6266 'headers': {'Content-Type': 'application/xml'},
6267 'body' : ''.concat(
6268 '<?xml version="1.0" encoding="utf-8"?>\n',
6269 '<doc>',
6270 (function(){
6271 escape = function(s){
6272 return s.replace(/"/g, '"')
6273 .replace(/>/g, '>')
6274 .replace(/</g, '<')
6275 .replace(/&/g, '&');
6276 };
6277 var content = '';
6278 for(var key in doc){
6279 if(!doc.hasOwnProperty(key)) continue;
6280 var value = escape(toJSON(doc[key]));
6281 var key = escape(key);
6282 content += ''.concat(
6283 '<' + key + '>',
6284 value
6285 '</' + key + '>'
6286 )
6287 }
6288 return content;
6289 })(),
6290 '</doc>'
6291 )
6292 }
6293 })
6294 registerType('text-json', 'text/json')
6295 provides('text-json', function(){
6296 return toJSON(doc);
6297 })
6298 }
6299 This function may return html, json , xml or our custom text json for‐
6301 mat representation of same document object with same processing rules.
6302 Probably, the xml provider in our function needs more care to handle
6303 nested objects correctly, and keys with invalid characters, but you’ve
6304 got the idea!
6305 SEE ALSO:
6307 CouchDB Guide:
6309 · Show Functions
6311 List Functions
6313 WARNING:
6314 List functions are deprecated in CouchDB 3.0, and will be removed in
6315 CouchDB 4.0.
6316 listfun(head, req)
6318 Arguments
6320 · head – view_head_info_object
6322 · req – Request object.
6324 Returns
6326 Last chunk.
6327 Return type
6329 string
6330 While Show Functions are used to customize document presentation, List
6332 Functions are used for the same purpose, but on View Functions results.
6333 The following list function formats the view and represents it as a
6335 very simple HTML page:
6336 function(head, req){
6338 start({
6339 'headers': {
6340 'Content-Type': 'text/html'
6341 }
6342 });
6343 send('<html><body><table>');
6344 send('<tr><th>ID</th><th>Key</th><th>Value</th></tr>');
6345 while(row = getRow()){
6346 send(''.concat(
6347 '<tr>',
6348 '<td>' + toJSON(row.id) + '</td>',
6349 '<td>' + toJSON(row.key) + '</td>',
6350 '<td>' + toJSON(row.value) + '</td>',
6351 '</tr>'
6352 ));
6353 }
6354 send('</table></body></html>');
6355 }
6356 Templates and styles could obviously be used to present data in a nicer
6358 fashion, but this is an excellent starting point. Note that you may
6359 also use registerType() and provides() functions in a similar way as
6360 for Show Functions! However, note that provides() expects the return
6361 value to be a string when used inside a list function, so you’ll need
6362 to use start() to set any custom headers and stringify your JSON before
6363 returning it.
6364 SEE ALSO:
6366 CouchDB Guide:
6368 · Transforming Views with List Functions
6370 Update Functions
6372 updatefun(doc, req)
6373 Arguments
6375 · doc – The document that is being processed.
6377 · req – request_object
6379 Returns
6381 Two-element array: the first element is the (updated or
6382 new) document, which is committed to the database. If the
6383 first element is null no document will be committed to
6384 the database. If you are updating an existing document,
6385 it should already have an _id set, and if you are creat‐
6386 ing a new document, make sure to set its _id to some‐
6387 thing, either generated based on the input or the
6388 req.uuid provided. The second element is the response
6389 that will be sent back to the caller.
6390 Update handlers are functions that clients can request to invoke
6392 server-side logic that will create or update a document. This feature
6393 allows a range of use cases such as providing a server-side last modi‐
6394 fied timestamp, updating individual fields in a document without first
6395 getting the latest revision, etc.
6396 When the request to an update handler includes a document ID in the
6398 URL, the server will provide the function with the most recent version
6399 of that document. You can provide any other values needed by the
6400 update handler function via the POST/PUT entity body or query string
6401 parameters of the request.
6402 A basic example that demonstrates all use-cases of update handlers:
6404 function(doc, req){
6406 if (!doc){
6407 if ('id' in req && req['id']){
6408 // create new document
6409 return [{'_id': req['id']}, 'New World']
6410 }
6411 // change nothing in database
6412 return [null, 'Empty World']
6413 }
6414 doc['world'] = 'hello';
6415 doc['edited_by'] = req['userCtx']['name']
6416 return [doc, 'Edited World!']
6417 }
6418 Filter Functions
6420 filterfun(doc, req)
6421 Arguments
6423 · doc – The document that is being processed
6425 · req – request_object
6427 Returns
6429 Boolean value: true means that doc passes the filter
6430 rules, false means that it does not.
6431 Filter functions mostly act like Show Functions and List Functions:
6433 they format, or filter the changes feed.
6434 Classic Filters
6436 By default the changes feed emits all database documents changes. But
6437 if you’re waiting for some special changes, processing all documents is
6438 inefficient.
6439 Filters are special design document functions that allow the changes
6441 feed to emit only specific documents that pass filter rules.
6442 Let’s assume that our database is a mailbox and we need to handle only
6444 new mail events (documents with the status new). Our filter function
6445 would look like this:
6446 function(doc, req){
6448 // we need only `mail` documents
6449 if (doc.type != 'mail'){
6450 return false;
6451 }
6452 // we're interested only in `new` ones
6453 if (doc.status != 'new'){
6454 return false;
6455 }
6456 return true; // passed!
6457 }
6458 Filter functions must return true if a document passed all the rules.
6460 Now, if you apply this function to the changes feed it will emit only
6461 changes about “new mails”:
6462 GET /somedatabase/_changes?filter=mailbox/new_mail HTTP/1.1
6464 {"results":[
6466 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoCqvJYgCRDA5ACKpxPWOUCiMr9hFUegKi8T1jlA4hKkDuzAC2yZRo","id":"df8eca9da37dade42ee4d7aa3401f1dd","changes":[{"rev":"1-c2e0085a21d34fa1cecb6dc26a4ae657"}]},
6467 {"seq":"9-g1AAAAIreJyVkEsKwjAURUMrqCOXoCuQ5MU0OrI70XyppcaRY92J7kR3ojupaSPUUgqWwAu85By4t0AITbJYo5k7aUNSAnyJ_SGFf4gEkvOyLPMsFtHRL8ZKaC1M0v3eq5ALP-X2a0G1xYKhgnONpmenjT04o_v5tOJ3LV5itTES_uP3FX9ppcAACaVsQAo38hNd_eVFt8ZklVljPqSPYLoH06PJhG0Cxq7-yhQcz-B4_fQCjFuqBjjewVF3E9cORoExSrpU_gHBTo5m","id":"df8eca9da37dade42ee4d7aa34024714","changes":[{"rev":"1-29d748a6e87b43db967fe338bcb08d74"}]},
6468 ],
6469 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6470 Note that the value of last_seq is 10-.., but we received only two
6472 records. Seems like any other changes were for documents that haven’t
6473 passed our filter.
6474 We probably need to filter the changes feed of our mailbox by more than
6476 a single status value. We’re also interested in statuses like “spam” to
6477 update spam-filter heuristic rules, “outgoing” to let a mail daemon
6478 actually send mails, and so on. Creating a lot of similar functions
6479 that actually do similar work isn’t good idea - so we need a dynamic
6480 filter.
6481 You may have noticed that filter functions take a second argument named
6483 request. This allows the creation of dynamic filters based on query
6484 parameters, user context and more.
6485 The dynamic version of our filter looks like this:
6487 function(doc, req){
6489 // we need only `mail` documents
6490 if (doc.type != 'mail'){
6491 return false;
6492 }
6493 // we're interested only in requested status
6494 if (doc.status != req.query.status){
6495 return false;
6496 }
6497 return true; // passed!
6498 }
6499 and now we have passed the status query parameter in the request to let
6501 our filter match only the required documents:
6502 GET /somedatabase/_changes?filter=mailbox/by_status&status=new HTTP/1.1
6504 {"results":[
6506 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoCqvJYgCRDA5ACKpxPWOUCiMr9hFUegKi8T1jlA4hKkDuzAC2yZRo","id":"df8eca9da37dade42ee4d7aa3401f1dd","changes":[{"rev":"1-c2e0085a21d34fa1cecb6dc26a4ae657"}]},
6507 {"seq":"9-g1AAAAIreJyVkEsKwjAURUMrqCOXoCuQ5MU0OrI70XyppcaRY92J7kR3ojupaSPUUgqWwAu85By4t0AITbJYo5k7aUNSAnyJ_SGFf4gEkvOyLPMsFtHRL8ZKaC1M0v3eq5ALP-X2a0G1xYKhgnONpmenjT04o_v5tOJ3LV5itTES_uP3FX9ppcAACaVsQAo38hNd_eVFt8ZklVljPqSPYLoH06PJhG0Cxq7-yhQcz-B4_fQCjFuqBjjewVF3E9cORoExSrpU_gHBTo5m","id":"df8eca9da37dade42ee4d7aa34024714","changes":[{"rev":"1-29d748a6e87b43db967fe338bcb08d74"}]},
6508 ],
6509 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6510 and we can easily change filter behavior with:
6512 GET /somedatabase/_changes?filter=mailbox/by_status&status=spam HTTP/1.1
6514 {"results":[
6516 {"seq":"6-g1AAAAIreJyVkM0JwjAYQD9bQT05gk4gaWIaPdlNNL_UUuPJs26im-gmuklMjVClFFoCXyDJe_BSAsA4jxVM7VHpJEswWyC_ktJfRBzEzDlX5DGPDv5gJLlSXKfN560KMfdTbL4W-FgM1oQzpmByskqbvdWqnc8qfvvHCyTXWuBu_K7iz38VCOOUENqjwg79hIvfvOhamQahROoVYn3-I5huwXSvm5BJsTbLTk3B8QiO58-_YMoMkT0cr-BwdRElmFKSNKniDcAcjmM","id":"8960e91220798fc9f9d29d24ed612e0d","changes":[{"rev":"3-cc6ff71af716ddc2ba114967025c0ee0"}]},
6517 ],
6518 "last_seq":"10-g1AAAAIreJyVkEsKwjAURR9tQR25BF2B5GMaHdmdaNIk1FLjyLHuRHeiO9Gd1LQRaimFlsALvOQcuLcAgGkWKpjbs9I4wYSvkDu4cA-BALkoyzLPQhGc3GKSCqWEjrvfexVy6abc_SxQWwzRVHCuYHaxSpuj1aqfTyp-3-IlSrdakmH8oeKvrRSIkJhSNiKFjdyEm7uc6N6YTKo3iI_pw5se3vRsMiETE23WgzJ5x8s73n-9EMYNTUc4Pt5RdxPVDkYJYxR3qfwLwW6OZw"}
6519 Combining filters with a continuous feed allows creating powerful
6521 event-driven systems.
6522 View Filters
6524 View filters are the same as classic filters above, with one small dif‐
6525 ference: they use the map instead of the filter function of a view, to
6526 filter the changes feed. Each time a key-value pair is emitted from the
6527 map function, a change is returned. This allows avoiding filter func‐
6528 tions that mostly do the same work as views.
6529 To use them just pass filter=_view and view=designdoc/viewname as
6531 request parameters to the changes feed:
6532 GET /somedatabase/_changes?filter=_view&view=dname/viewname HTTP/1.1
6534 NOTE:
6536 Since view filters use map functions as filters, they can’t show any
6537 dynamic behavior since request object is not available.
6538 SEE ALSO:
6540 CouchDB Guide:
6542 · Guide to filter change notification
6544 Validate Document Update Functions
6546 validatefun(newDoc, oldDoc, userCtx, secObj)
6547 Arguments
6549 · newDoc – New version of document that will be stored.
6551 · oldDoc – Previous version of document that is already
6553 stored.
6554 · userCtx – userctx_object
6556 · secObj – security_object
6558 Throws forbidden error to gracefully prevent document storing.
6560 Throws unauthorized error to prevent storage and allow the user
6562 to re-auth.
6563 A design document may contain a function named validate_doc_update
6565 which can be used to prevent invalid or unauthorized document update
6566 requests from being stored. The function is passed the new document
6567 from the update request, the current document stored in the database, a
6568 userctx_object containing information about the user writing the docu‐
6569 ment (if present), and a security_object with lists of database secu‐
6570 rity roles.
6571 Validation functions typically examine the structure of the new docu‐
6573 ment to ensure that required fields are present and to verify that the
6574 requesting user should be allowed to make changes to the document prop‐
6575 erties. For example, an application may require that a user must be
6576 authenticated in order to create a new document or that specific docu‐
6577 ment fields be present when a document is updated. The validation func‐
6578 tion can abort the pending document write by throwing one of two error
6579 objects:
6580 // user is not authorized to make the change but may re-authenticate
6582 throw({ unauthorized: 'Error message here.' });
6583 // change is not allowed
6585 throw({ forbidden: 'Error message here.' });
6586 Document validation is optional, and each design document in the data‐
6588 base may have at most one validation function. When a write request is
6589 received for a given database, the validation function in each design
6590 document in that database is called in an unspecified order. If any of
6591 the validation functions throw an error, the write will not succeed.
6592 Example: The _design/_auth ddoc from _users database uses a validation
6594 function to ensure that documents contain some required fields and are
6595 only modified by a user with the _admin role:
6596 function(newDoc, oldDoc, userCtx, secObj) {
6598 if (newDoc._deleted === true) {
6599 // allow deletes by admins and matching users
6600 // without checking the other fields
6601 if ((userCtx.roles.indexOf('_admin') !== -1) ||
6602 (userCtx.name == oldDoc.name)) {
6603 return;
6604 } else {
6605 throw({forbidden: 'Only admins may delete other user docs.'});
6606 }
6607 }
6608 if ((oldDoc && oldDoc.type !== 'user') || newDoc.type !== 'user') {
6610 throw({forbidden : 'doc.type must be user'});
6611 } // we only allow user docs for now
6612 if (!newDoc.name) {
6614 throw({forbidden: 'doc.name is required'});
6615 }
6616 if (!newDoc.roles) {
6618 throw({forbidden: 'doc.roles must exist'});
6619 }
6620 if (!isArray(newDoc.roles)) {
6622 throw({forbidden: 'doc.roles must be an array'});
6623 }
6624 if (newDoc._id !== ('org.couchdb.user:' + newDoc.name)) {
6626 throw({
6627 forbidden: 'Doc ID must be of the form org.couchdb.user:name'
6628 });
6629 }
6630 if (oldDoc) { // validate all updates
6632 if (oldDoc.name !== newDoc.name) {
6633 throw({forbidden: 'Usernames can not be changed.'});
6634 }
6635 }
6636 if (newDoc.password_sha && !newDoc.salt) {
6638 throw({
6639 forbidden: 'Users with password_sha must have a salt.' +
6640 'See /_utils/script/couch.js for example code.'
6641 });
6642 }
6643 var is_server_or_database_admin = function(userCtx, secObj) {
6645 // see if the user is a server admin
6646 if(userCtx.roles.indexOf('_admin') !== -1) {
6647 return true; // a server admin
6648 }
6649 // see if the user a database admin specified by name
6651 if(secObj && secObj.admins && secObj.admins.names) {
6652 if(secObj.admins.names.indexOf(userCtx.name) !== -1) {
6653 return true; // database admin
6654 }
6655 }
6656 // see if the user a database admin specified by role
6658 if(secObj && secObj.admins && secObj.admins.roles) {
6659 var db_roles = secObj.admins.roles;
6660 for(var idx = 0; idx < userCtx.roles.length; idx++) {
6661 var user_role = userCtx.roles[idx];
6662 if(db_roles.indexOf(user_role) !== -1) {
6663 return true; // role matches!
6664 }
6665 }
6666 }
6667 return false; // default to no admin
6669 }
6670 if (!is_server_or_database_admin(userCtx, secObj)) {
6672 if (oldDoc) { // validate non-admin updates
6673 if (userCtx.name !== newDoc.name) {
6674 throw({
6675 forbidden: 'You may only update your own user document.'
6676 });
6677 }
6678 // validate role updates
6679 var oldRoles = oldDoc.roles.sort();
6680 var newRoles = newDoc.roles.sort();
6681 if (oldRoles.length !== newRoles.length) {
6683 throw({forbidden: 'Only _admin may edit roles'});
6684 }
6685 for (var i = 0; i < oldRoles.length; i++) {
6687 if (oldRoles[i] !== newRoles[i]) {
6688 throw({forbidden: 'Only _admin may edit roles'});
6689 }
6690 }
6691 } else if (newDoc.roles.length > 0) {
6692 throw({forbidden: 'Only _admin may set roles'});
6693 }
6694 }
6695 // no system roles in users db
6697 for (var i = 0; i < newDoc.roles.length; i++) {
6698 if (newDoc.roles[i][0] === '_') {
6699 throw({
6700 forbidden:
6701 'No system roles (starting with underscore) in users db.'
6702 });
6703 }
6704 }
6705 // no system names as names
6707 if (newDoc.name[0] === '_') {
6708 throw({forbidden: 'Username may not start with underscore.'});
6709 }
6710 var badUserNameChars = [':'];
6712 for (var i = 0; i < badUserNameChars.length; i++) {
6714 if (newDoc.name.indexOf(badUserNameChars[i]) >= 0) {
6715 throw({forbidden: 'Character `' + badUserNameChars[i] +
6716 '` is not allowed in usernames.'});
6717 }
6718 }
6719 }
6720 NOTE:
6722 The return statement is used only for function, it has no impact on
6723 the validation process.
6724 SEE ALSO:
6726 CouchDB Guide:
6728 · Validation Functions
6730 Guide to Views
6732 Views are the primary tool used for querying and reporting on CouchDB
6733 documents. There you’ll learn how they works and how to use them to
6734 build effective applications with CouchDB
6735 Introduction to Views
6737 Views are useful for many purposes:
6738 · Filtering the documents in your database to find those relevant to a
6740 particular process.
6741 · Extracting data from your documents and presenting it in a specific
6743 order.
6744 · Building efficient indexes to find documents by any value or struc‐
6746 ture that resides in them.
6747 · Use these indexes to represent relationships among documents.
6749 · Finally, with views you can make all sorts of calculations on the
6751 data in your documents. For example, if documents represent your com‐
6752 pany’s financial transactions, a view can answer the question of what
6753 the spending was in the last week, month, or year.
6754 What Is a View?
6756 Let’s go through the different use cases. First is extracting data that
6757 you might need for a special purpose in a specific order. For a front
6758 page, we want a list of blog post titles sorted by date. We’ll work
6759 with a set of example documents as we walk through how views work:
6760 {
6762 "_id":"biking",
6763 "_rev":"AE19EBC7654",
6764 "title":"Biking",
6766 "body":"My biggest hobby is mountainbiking. The other day...",
6767 "date":"2009/01/30 18:04:11"
6768 }
6769 {
6771 "_id":"bought-a-cat",
6772 "_rev":"4A3BBEE711",
6773 "title":"Bought a Cat",
6775 "body":"I went to the the pet store earlier and brought home a little kitty...",
6776 "date":"2009/02/17 21:13:39"
6777 }
6778 {
6780 "_id":"hello-world",
6781 "_rev":"43FBA4E7AB",
6782 "title":"Hello World",
6784 "body":"Well hello and welcome to my new blog...",
6785 "date":"2009/01/15 15:52:20"
6786 }
6787 Three will do for the example. Note that the documents are sorted by
6789 “_id”, which is how they are stored in the database. Now we define a
6790 view. Bear with us without an explanation while we show you some code:
6791 function(doc) {
6793 if(doc.date && doc.title) {
6794 emit(doc.date, doc.title);
6795 }
6796 }
6797 This is a map function, and it is written in JavaScript. If you are not
6799 familiar with JavaScript but have used C or any other C-like language
6800 such as Java, PHP, or C#, this should look familiar. It is a simple
6801 function definition.
6802 You provide CouchDB with view functions as strings stored inside the
6804 views field of a design document. You don’t run it yourself. Instead,
6805 when you query your view, CouchDB takes the source code and runs it for
6806 you on every document in the database your view was defined in. You
6807 query your view to retrieve the view result.
6808 All map functions have a single parameter doc. This is a single docu‐
6810 ment in the database. Our map function checks whether our document has
6811 a date and a title attribute — luckily, all of our documents have them
6812 — and then calls the built-in emit() function with these two attributes
6813 as arguments.
6814 The emit() function always takes two arguments: the first is key, and
6816 the second is value. The emit(key, value) function creates an entry in
6817 our view result. One more thing: the emit() function can be called mul‐
6818 tiple times in the map function to create multiple entries in the view
6819 results from a single document, but we are not doing that yet.
6820 CouchDB takes whatever you pass into the emit() function and puts it
6822 into a list (see Table 1, “View results” below). Each row in that list
6823 includes the key and value. More importantly, the list is sorted by key
6824 (by doc.date in our case). The most important feature of a view result
6825 is that it is sorted by key. We will come back to that over and over
6826 again to do neat things. Stay tuned.
6827 Table 1. View results:
6829 ┌──────────────────────┬────────────────┐
6831 │Key │ Value │
6832 ├──────────────────────┼────────────────┤
6833 │“2009/01/15 15:52:20” │ “Hello World” │
6834 ├──────────────────────┼────────────────┤
6835 │“2009/01/30 18:04:11” │ “Biking” │
6836 ├──────────────────────┼────────────────┤
6837 │“2009/02/17 21:13:39” │ “Bought a Cat” │
6838 └──────────────────────┴────────────────┘
6839 When you query your view, CouchDB takes the source code and runs it for
6841 you on every document in the database. If you have a lot of documents,
6842 that takes quite a bit of time and you might wonder if it is not horri‐
6843 bly inefficient to do this. Yes, it would be, but CouchDB is designed
6844 to avoid any extra costs: it only runs through all documents once, when
6845 you first query your view. If a document is changed, the map function
6846 is only run once, to recompute the keys and values for that single doc‐
6847 ument.
6848 The view result is stored in a B-tree, just like the structure that is
6850 responsible for holding your documents. View B-trees are stored in
6851 their own file, so that for high-performance CouchDB usage, you can
6852 keep views on their own disk. The B-tree provides very fast lookups of
6853 rows by key, as well as efficient streaming of rows in a key range. In
6854 our example, a single view can answer all questions that involve time:
6855 “Give me all the blog posts from last week” or “last month” or “this
6856 year.” Pretty neat.
6857 When we query our view, we get back a list of all documents sorted by
6859 date. Each row also includes the post title so we can construct links
6860 to posts. Table 1 is just a graphical representation of the view
6861 result. The actual result is JSON-encoded and contains a little more
6862 metadata:
6863 {
6865 "total_rows": 3,
6866 "offset": 0,
6867 "rows": [
6868 {
6869 "key": "2009/01/15 15:52:20",
6870 "id": "hello-world",
6871 "value": "Hello World"
6872 },
6873 {
6875 "key": "2009/01/30 18:04:11",
6876 "id": "biking",
6877 "value": "Biking"
6878 },
6879 {
6881 "key": "2009/02/17 21:13:39",
6882 "id": "bought-a-cat",
6883 "value": "Bought a Cat"
6884 }
6885 ]
6887 }
6888 Now, the actual result is not as nicely formatted and doesn’t include
6890 any superfluous whitespace or newlines, but this is better for you (and
6891 us!) to read and understand. Where does that “id” member in the result
6892 rows come from? That wasn’t there before. That’s because we omitted it
6893 earlier to avoid confusion. CouchDB automatically includes the document
6894 ID of the document that created the entry in the view result. We’ll use
6895 this as well when constructing links to the blog post pages.
6896 WARNING:
6898 Do not emit the entire document as the value of your emit(key,
6899 value) statement unless you’re sure you know you want it. This
6900 stores an entire additional copy of your document in the view’s sec‐
6901 ondary index. Views with emit(key, doc) take longer to update,
6902 longer to write to disk, and consume significantly more disk space.
6903 The only advantage is that they are faster to query than using the
6904 ?include_docs=true parameter when querying a view.
6905 Consider the trade-offs before emitting the entire document. Often
6907 it is sufficient to emit only a portion of the document, or just a
6908 single key / value pair, in your views.
6909 Efficient Lookups
6911 Let’s move on to the second use case for views: “building efficient
6912 indexes to find documents by any value or structure that resides in
6913 them.” We already explained the efficient indexing, but we skipped a
6914 few details. This is a good time to finish this discussion as we are
6915 looking at map functions that are a little more complex.
6916 First, back to the B-trees! We explained that the B-tree that backs the
6918 key-sorted view result is built only once, when you first query a view,
6919 and all subsequent queries will just read the B-tree instead of execut‐
6920 ing the map function for all documents again. What happens, though,
6921 when you change a document, add a new one, or delete one? Easy: CouchDB
6922 is smart enough to find the rows in the view result that were created
6923 by a specific document. It marks them invalid so that they no longer
6924 show up in view results. If the document was deleted, we’re good — the
6925 resulting B-tree reflects the state of the database. If a document got
6926 updated, the new document is run through the map function and the
6927 resulting new lines are inserted into the B-tree at the correct spots.
6928 New documents are handled in the same way. The B-tree is a very effi‐
6929 cient data structure for our needs, and the crash-only design of
6930 CouchDB databases is carried over to the view indexes as well.
6931 To add one more point to the efficiency discussion: usually multiple
6933 documents are updated between view queries. The mechanism explained in
6934 the previous paragraph gets applied to all changes in the database
6935 since the last time the view was queried in a batch operation, which
6936 makes things even faster and is generally a better use of your
6937 resources.
6938 Find One
6940 On to more complex map functions. We said “find documents by any value
6941 or structure that resides in them.” We already explained how to extract
6942 a value by which to sort a list of views (our date field). The same
6943 mechanism is used for fast lookups. The URI to query to get a view’s
6944 result is /database/_design/designdocname/_view/viewname. This gives
6945 you a list of all rows in the view. We have only three documents, so
6946 things are small, but with thousands of documents, this can get long.
6947 You can add view parameters to the URI to constrain the result set. Say
6948 we know the date of a blog post. To find a single document, we would
6949 use /blog/_design/docs/_view/by_date?key="2009/01/30 18:04:11" to get
6950 the “Biking” blog post. Remember that you can place whatever you like
6951 in the key parameter to the emit() function. Whatever you put in there,
6952 we can now use to look up exactly — and fast.
6953 Note that in the case where multiple rows have the same key (perhaps we
6955 design a view where the key is the name of the post’s author), key
6956 queries can return more than one row.
6957 Find Many
6959 We talked about “getting all posts for last month.” If it’s February
6960 now, this is as easy as:
6961 /blog/_design/docs/_view/by_date?startkey="2010/01/01 00:00:00"&endkey="2010/02/00 00:00:00"
6963 The startkey and endkey parameters specify an inclusive range on which
6965 we can search.
6966 To make things a little nicer and to prepare for a future example, we
6968 are going to change the format of our date field. Instead of a string,
6969 we are going to use an array, where individual members are part of a
6970 timestamp in decreasing significance. This sounds fancy, but it is
6971 rather easy. Instead of:
6972 {
6974 "date": "2009/01/31 00:00:00"
6975 }
6976 we use:
6978 {
6980 "date": [2009, 1, 31, 0, 0, 0]
6981 }
6982 Our map function does not have to change for this, but our view result
6984 looks a little different:
6985 Table 2. New view results:
6987 ┌──────────────────────────┬────────────────┐
6989 │Key │ Value │
6990 ├──────────────────────────┼────────────────┤
6991 │[2009, 1, 15, 15, 52, 20] │ “Hello World” │
6992 ├──────────────────────────┼────────────────┤
6993 │[2009, 2, 17, 21, 13, 39] │ “Biking” │
6994 ├──────────────────────────┼────────────────┤
6995 │[2009, 1, 30, 18, 4, 11] │ “Bought a Cat” │
6996 └──────────────────────────┴────────────────┘
6997 And our queries change to:
6999 /blog/_design/docs/_view/by_date?startkey=[2010, 1, 1, 0, 0, 0]&endkey=[2010, 2, 1, 0, 0, 0]
7001 For all you care, this is just a change in syntax, not meaning. But it
7003 shows you the power of views. Not only can you construct an index with
7004 scalar values like strings and integers, you can also use JSON struc‐
7005 tures as keys for your views. Say we tag our documents with a list of
7006 tags and want to see all tags, but we don’t care for documents that
7007 have not been tagged.
7008 {
7010 ...
7011 tags: ["cool", "freak", "plankton"],
7012 ...
7013 }
7014 {
7016 ...
7017 tags: [],
7018 ...
7019 }
7020 function(doc) {
7022 if(doc.tags.length > 0) {
7023 for(var idx in doc.tags) {
7024 emit(doc.tags[idx], null);
7025 }
7026 }
7027 }
7028 This shows a few new things. You can have conditions on structure
7030 (if(doc.tags.length > 0)) instead of just values. This is also an exam‐
7031 ple of how a map function calls emit() multiple times per document.
7032 And finally, you can pass null instead of a value to the value parame‐
7033 ter. The same is true for the key parameter. We’ll see in a bit how
7034 that is useful.
7035 Reversed Results
7037 To retrieve view results in reverse order, use the descending=true
7038 query parameter. If you are using a startkey parameter, you will find
7039 that CouchDB returns different rows or no rows at all. What’s up with
7040 that?
7041 It’s pretty easy to understand when you see how view query options work
7043 under the hood. A view is stored in a tree structure for fast lookups.
7044 Whenever you query a view, this is how CouchDB operates:
7045 1. Starts reading at the top, or at the position that startkey speci‐
7047 fies, if present.
7048 2. Returns one row at a time until the end or until it hits endkey, if
7050 present.
7051 If you specify descending=true, the reading direction is reversed, not
7053 the sort order of the rows in the view. In addition, the same two-step
7054 procedure is followed.
7055 Say you have a view result that looks like this:
7057 ┌────┬───────┐
7059 │Key │ Value │
7060 ├────┼───────┤
7061 │0 │ “foo” │
7062 ├────┼───────┤
7063 │1 │ “bar” │
7064 ├────┼───────┤
7065 │2 │ “baz” │
7066 └────┴───────┘
7067 Here are potential query options: ?startkey=1&descending=true. What
7069 will CouchDB do? See #1 above: it jumps to startkey, which is the row
7070 with the key 1, and starts reading backward until it hits the end of
7071 the view. So the particular result would be:
7072 ┌────┬───────┐
7074 │Key │ Value │
7075 ├────┼───────┤
7076 │1 │ “bar” │
7077 ├────┼───────┤
7078 │0 │ “foo” │
7079 └────┴───────┘
7080 This is very likely not what you want. To get the rows with the indexes
7082 1 and 2 in reverse order, you need to switch the startkey to endkey:
7083 endkey=1&descending=true:
7084 ┌────┬───────┐
7086 │Key │ Value │
7087 ├────┼───────┤
7088 │2 │ “baz” │
7089 └────┴───────┘
7090 │1 │ “bar” │
7092 └────┴───────┘
7093 Now that looks a lot better. CouchDB started reading at the bottom of
7095 the view and went backward until it hit endkey.
7096 The View to Get Comments for Posts
7098 We use an array key here to support the group_level reduce query param‐
7099 eter. CouchDB’s views are stored in the B-tree file structure. Because
7100 of the way B-trees are structured, we can cache the intermediate reduce
7101 results in the non-leaf nodes of the tree, so reduce queries can be
7102 computed along arbitrary key ranges in logarithmic time. See Figure 1,
7103 “Comments map function”.
7104 In the blog app, we use group_level reduce queries to compute the count
7106 of comments both on a per-post and total basis, achieved by querying
7107 the same view index with different methods. With some array keys, and
7108 assuming each key has the value 1:
7109 ["a","b","c"]
7111 ["a","b","e"]
7112 ["a","c","m"]
7113 ["b","a","c"]
7114 ["b","a","g"]
7115 the reduce view:
7117 function(keys, values, rereduce) {
7119 return sum(values)
7120 }
7121 or:
7123 _sum
7125 which is a built-in CouchDB reduce function (the others are _count and
7127 _stats). _sum here returns the total number of rows between the start
7128 and end key. So with startkey=["a","b"]&endkey=["b"] (which includes
7129 the first three of the above keys) the result would equal 3. The effect
7130 is to count rows. If you’d like to count rows without depending on the
7131 row value, you can switch on the rereduce parameter:
7132 function(keys, values, rereduce) {
7134 if (rereduce) {
7135 return sum(values);
7136 } else {
7137 return values.length;
7138 }
7139 }
7140 NOTE:
7142 The JavaScript function above could be effectively replaced by the
7143 built-in _count.
7144 [image: Comments map function] [image] Figure 1. Comments map func‐
7145 tion.UNINDENT
7146 This is the reduce view used by the example app to count comments,
7148 while utilizing the map to output the comments, which are more useful
7149 than just 1 over and over. It pays to spend some time playing around
7150 with map and reduce functions. Fauxton is OK for this, but it doesn’t
7151 give full access to all the query parameters. Writing your own test
7152 code for views in your language of choice is a great way to explore
7153 the nuances and capabilities of CouchDB’s incremental MapReduce sys‐
7154 tem.
7155 Anyway, with a group_level query, you’re basically running a series
7157 of reduce range queries: one for each group that shows up at the
7158 level you query. Let’s reprint the key list from earlier, grouped at
7159 level 1:
7160 ["a"] 3
7162 ["b"] 2
7163 And at group_level=2:
7165 ["a","b"] 2
7167 ["a","c"] 1
7168 ["b","a"] 2
7169 Using the parameter group=true makes it behave as though it were
7171 group_level=999, so in the case of our current example, it would give
7172 the number 1 for each key, as there are no exactly duplicated keys.
7173 Reduce/Rereduce
7175 We briefly talked about the rereduce parameter to the reduce function.
7176 We’ll explain what’s up with it in this section. By now, you should
7177 have learned that your view result is stored in B-tree index structure
7178 for efficiency. The existence and use of the rereduce parameter is
7179 tightly coupled to how the B-tree index works.
7180 Consider the map result are:
7182 "afrikaans", 1
7184 "afrikaans", 1
7185 "chinese", 1
7186 "chinese", 1
7187 "chinese", 1
7188 "chinese", 1
7189 "french", 1
7190 "italian", 1
7191 "italian", 1
7192 "spanish", 1
7193 "vietnamese", 1
7194 "vietnamese", 1
7195 Example 1. Example view result (mmm, food)
7197 When we want to find out how many dishes there are per origin, we can
7199 reuse the simple reduce function shown earlier:
7200 function(keys, values, rereduce) {
7202 return sum(values);
7203 }
7204 Figure 2, “The B-tree index” shows a simplified version of what the
7206 B-tree index looks like. We abbreviated the key strings.
7207 [image: The B-tree index] [image] Figure 2. The B-tree index.UNINDENT
7208 The view result is what computer science grads call a “pre-order”
7210 walk through the tree. We look at each element in each node starting
7211 from the left. Whenever we see that there is a subnode to descend
7212 into, we descend and start reading the elements in that subnode. When
7213 we have walked through the entire tree, we’re done.
7214 You can see that CouchDB stores both keys and values inside each leaf
7216 node. In our case, it is simply always 1, but you might have a value
7217 where you count other results and then all rows have a different
7218 value. What’s important is that CouchDB runs all elements that are
7219 within a node into the reduce function (setting the rereduce parame‐
7220 ter to false) and stores the result inside the parent node along with
7221 the edge to the subnode. In our case, each edge has a 3 representing
7222 the reduce value for the node it points to.
7223 NOTE:
7225 In reality, nodes have more than 1,600 elements in them. CouchDB
7226 computes the result for all the elements in multiple iterations over
7227 the elements in a single node, not all at once (which would be dis‐
7228 astrous for memory consumption).
7229 Now let’s see what happens when we run a query. We want to know how
7231 many “chinese” entries we have. The query option is simple: ?key="chi‐
7232 nese". See Figure 3, “The B-tree index reduce result”.
7233 [image: The B-tree index reduce result] [image] Figure 3. The B-tree
7234 index reduce result.UNINDENT
7235 CouchDB detects that all values in the subnode include the “chinese”
7237 key. It concludes that it can take just the 3 values associated with
7238 that node to compute the final result. It then finds the node left to
7239 it and sees that it’s a node with keys outside the requested range
7240 (key= requests a range where the beginning and the end are the same
7241 value). It concludes that it has to use the “chinese” element’s value
7242 and the other node’s value and run them through the reduce function
7243 with the rereduce parameter set to true.
7244 The reduce function effectively calculates 3 + 1 at query time and
7246 returns the desired result. The next example shows some pseudocode
7247 that shows the last invocation of the reduce function with actual
7248 values:
7249 function(null, [3, 1], true) {
7251 return sum([3, 1]);
7252 }
7253 Now, we said your reduce function must actually reduce your values. If
7255 you see the B-tree, it should become obvious what happens when you
7256 don’t reduce your values. Consider the following map result and reduce
7257 function. This time we want to get a list of all the unique labels in
7258 our view:
7259 "abc", "afrikaans"
7261 "cef", "afrikaans"
7262 "fhi", "chinese"
7263 "hkl", "chinese"
7264 "ino", "chinese"
7265 "lqr", "chinese"
7266 "mtu", "french"
7267 "owx", "italian"
7268 "qza", "italian"
7269 "tdx", "spanish"
7270 "xfg", "vietnamese"
7271 "zul", "vietnamese"
7272 We don’t care for the key here and only list all the labels we have.
7274 Our reduce function removes duplicates:
7275 function(keys, values, rereduce) {
7277 var unique_labels = {};
7278 values.forEach(function(label) {
7279 if(!unique_labels[label]) {
7280 unique_labels[label] = true;
7281 }
7282 });
7283 return unique_labels;
7285 }
7286 This translates to Figure 4, “An overflowing reduce index”.
7288 We hope you get the picture. The way the B-tree storage works means
7290 that if you don’t actually reduce your data in the reduce function, you
7291 end up having CouchDB copy huge amounts of data around that grow lin‐
7292 early, if not faster, with the number of rows in your view.
7293 CouchDB will be able to compute the final result, but only for views
7295 with a few rows. Anything larger will experience a ridiculously slow
7296 view build time. To help with that, CouchDB since version 0.10.0 will
7297 throw an error if your reduce function does not reduce its input val‐
7298 ues.
7299 [image: An overflowing reduce index] [image] Figure 4. An overflowing
7300 reduce index.UNINDENT
7301 One vs. Multiple Design Documents
7303 A common question is: when should I split multiple views into multiple
7304 design documents, or keep them together?
7305 Each view you create corresponds to one B-tree. All views in a single
7307 design document will live in the same set of index files on disk (one
7308 file per database shard; in 2.0+ by default, 8 files per node).
7309 The most practical consideration for separating views into separate
7311 documents is how often you change those views. Views that change often,
7312 and are in the same design document as other views, will invalidate
7313 those other views’ indexes when the design document is written, forcing
7314 them all to rebuild from scratch. Obviously you will want to avoid this
7315 in production!
7316 However, when you have multiple views with the same map function in the
7318 same design document, CouchDB will optimize and only calculate that map
7319 function once. This lets you have two views with different reduce func‐
7320 tions (say, one with _sum and one with _stats) but build only a single
7321 copy of the mapped index. It also saves disk space and the time to
7322 write multiple copies to disk.
7323 Another benefit of having multiple views in the same design document is
7325 that the index files can keep a single index of backwards references
7326 from docids to rows. CouchDB needs these “back refs” to invalidate rows
7327 in a view when a document is deleted (otherwise, a delete would force a
7328 total rebuild!)
7329 One other consideration is that each separate design document will
7331 spawn another (set of) couchjs processes to generate the view, one per
7332 shard. Depending on the number of cores on your server(s), this may be
7333 efficient (using all of the idle cores you have) or inefficient (over‐
7334 loading the CPU on your servers). The exact situation will depend on
7335 your deployment architecture.
7336 So, should you use one or multiple design documents? The choice is
7338 yours.
7339 Lessons Learned
7341 · If you don’t use the key field in the map function, you are probably
7342 doing it wrong.
7343 · If you are trying to make a list of values unique in the reduce func‐
7345 tions, you are probably doing it wrong.
7346 · If you don’t reduce your values to a single scalar value or a small
7348 fixed-sized object or array with a fixed number of scalar values of
7349 small sizes, you are probably doing it wrong.
7350 Wrapping Up
7352 Map functions are side effect–free functions that take a document as
7353 argument and emit key/value pairs. CouchDB stores the emitted rows by
7354 constructing a sorted B-tree index, so row lookups by key, as well as
7355 streaming operations across a range of rows, can be accomplished in a
7356 small memory and processing footprint, while writes avoid seeks. Gener‐
7357 ating a view takes O(N), where N is the total number of rows in the
7358 view. However, querying a view is very quick, as the B-tree remains
7359 shallow even when it contains many, many keys.
7360 Reduce functions operate on the sorted rows emitted by map view func‐
7362 tions. CouchDB’s reduce functionality takes advantage of one of the
7363 fundamental properties of B-tree indexes: for every leaf node (a sorted
7364 row), there is a chain of internal nodes reaching back to the root.
7365 Each leaf node in the B-tree carries a few rows (on the order of tens,
7366 depending on row size), and each internal node may link to a few leaf
7367 nodes or other internal nodes.
7368 The reduce function is run on every node in the tree in order to calcu‐
7370 late the final reduce value. The end result is a reduce function that
7371 can be incrementally updated upon changes to the map function, while
7372 recalculating the reduction values for a minimum number of nodes. The
7373 initial reduction is calculated once per each node (inner and leaf) in
7374 the tree.
7375 When run on leaf nodes (which contain actual map rows), the reduce
7377 function’s third parameter, rereduce, is false. The arguments in this
7378 case are the keys and values as output by the map function. The func‐
7379 tion has a single returned reduction value, which is stored on the
7380 inner node that a working set of leaf nodes have in common, and is used
7381 as a cache in future reduce calculations.
7382 When the reduce function is run on inner nodes, the rereduce flag is
7384 true. This allows the function to account for the fact that it will be
7385 receiving its own prior output. When rereduce is true, the values
7386 passed to the function are intermediate reduction values as cached from
7387 previous calculations. When the tree is more than two levels deep, the
7388 rereduce phase is repeated, consuming chunks of the previous level’s
7389 output until the final reduce value is calculated at the root node.
7390 A common mistake new CouchDB users make is attempting to construct com‐
7392 plex aggregate values with a reduce function. Full reductions should
7393 result in a scalar value, like 5, and not, for instance, a JSON hash
7394 with a set of unique keys and the count of each. The problem with this
7395 approach is that you’ll end up with a very large final value. The num‐
7396 ber of unique keys can be nearly as large as the number of total keys,
7397 even for a large set. It is fine to combine a few scalar calculations
7398 into one reduce function; for instance, to find the total, average, and
7399 standard deviation of a set of numbers in a single function.
7400 If you’re interested in pushing the edge of CouchDB’s incremental
7402 reduce functionality, have a look at Google’s paper on Sawzall, which
7403 gives examples of some of the more exotic reductions that can be accom‐
7404 plished in a system with similar constraints.
7405 Views Collation
7407 Basics
7408 View functions specify a key and a value to be returned for each row.
7409 CouchDB collates the view rows by this key. In the following example,
7410 the LastName property serves as the key, thus the result will be sorted
7411 by LastName:
7412 function(doc) {
7414 if (doc.Type == "customer") {
7415 emit(doc.LastName, {FirstName: doc.FirstName, Address: doc.Address});
7416 }
7417 }
7418 CouchDB allows arbitrary JSON structures to be used as keys. You can
7420 use JSON arrays as keys for fine-grained control over sorting and
7421 grouping.
7422 Examples
7424 The following clever trick would return both customer and order docu‐
7425 ments. The key is composed of a customer _id and a sorting token.
7426 Because the key for order documents begins with the _id of a customer
7427 document, all the orders will be sorted by customer. Because the sort‐
7428 ing token for customers is lower than the token for orders, the cus‐
7429 tomer document will come before the associated orders. The values 0 and
7430 1 for the sorting token are arbitrary.
7431 function(doc) {
7433 if (doc.Type == "customer") {
7434 emit([doc._id, 0], null);
7435 } else if (doc.Type == "order") {
7436 emit([doc.customer_id, 1], null);
7437 }
7438 }
7439 To list a specific customer with _id XYZ, and all of that customer’s
7441 orders, limit the startkey and endkey ranges to cover only documents
7442 for that customer’s _id:
7443 startkey=["XYZ"]&endkey=["XYZ", {}]
7445 It is not recommended to emit the document itself in the view. Instead,
7447 to include the bodies of the documents when requesting the view,
7448 request the view with ?include_docs=true.
7449 Sorting by Dates
7451 It maybe be convenient to store date attributes in a human readable
7452 format (i.e. as a string), but still sort by date. This can be done by
7453 converting the date to a number in the emit() function. For example,
7454 given a document with a created_at attribute of 'Wed Jul 23 16:29:21
7455 +0100 2013', the following emit function would sort by date:
7456 emit(Date.parse(doc.created_at).getTime(), null);
7458 Alternatively, if you use a date format which sorts lexicographically,
7460 such as "2013/06/09 13:52:11 +0000" you can just
7461 emit(doc.created_at, null);
7463 and avoid the conversion. As a bonus, this date format is compatible
7465 with the JavaScript date parser, so you can use new Date(doc.cre‐
7466 ated_at) in your client side JavaScript to make date sorting easy in
7467 the browser.
7468 String Ranges
7470 If you need start and end keys that encompass every string with a given
7471 prefix, it is better to use a high value Unicode character, than to use
7472 a 'ZZZZ' suffix.
7473 That is, rather than:
7475 startkey="abc"&endkey="abcZZZZZZZZZ"
7477 You should use:
7479 startkey="abc"&endkey="abc\ufff0"
7481 Collation Specification
7483 This section is based on the view_collation function in
7484 view_collation.js:
7485 // special values sort before all other types
7487 null
7488 false
7489 true
7490 // then numbers
7492 1
7493 2
7494 3.0
7495 4
7496 // then text, case sensitive
7498 "a"
7499 "A"
7500 "aa"
7501 "b"
7502 "B"
7503 "ba"
7504 "bb"
7505 // then arrays. compared element by element until different.
7507 // Longer arrays sort after their prefixes
7508 ["a"]
7509 ["b"]
7510 ["b","c"]
7511 ["b","c", "a"]
7512 ["b","d"]
7513 ["b","d", "e"]
7514 // then object, compares each key value in the list until different.
7516 // larger objects sort after their subset objects.
7517 {a:1}
7518 {a:2}
7519 {b:1}
7520 {b:2}
7521 {b:2, a:1} // Member order does matter for collation.
7522 // CouchDB preserves member order
7523 // but doesn't require that clients will.
7524 // this test might fail if used with a js engine
7525 // that doesn't preserve order
7526 {b:2, c:2}
7527 Comparison of strings is done using ICU which implements the Unicode
7529 Collation Algorithm, giving a dictionary sorting of keys. This can
7530 give surprising results if you were expecting ASCII ordering. Note
7531 that:
7532 · All symbols sort before numbers and letters (even the “high” symbols
7534 like tilde, 0x7e)
7535 · Differing sequences of letters are compared without regard to case,
7537 so a < aa but also A < aa and a < AA
7538 · Identical sequences of letters are compared with regard to case, with
7540 lowercase before uppercase, so a < A
7541 You can demonstrate the collation sequence for 7-bit ASCII characters
7543 like this:
7544 require 'rubygems'
7546 require 'restclient'
7547 require 'json'
7548 DB="http://127.0.0.1:5984/collator"
7550 RestClient.delete DB rescue nil
7552 RestClient.put "#{DB}",""
7553 (32..126).each do |c|
7555 RestClient.put "#{DB}/#{c.to_s(16)}", {"x"=>c.chr}.to_json
7556 end
7557 RestClient.put "#{DB}/_design/test", <<EOS
7559 {
7560 "views":{
7561 "one":{
7562 "map":"function (doc) { emit(doc.x,null); }"
7563 }
7564 }
7565 }
7566 EOS
7567 puts RestClient.get("#{DB}/_design/test/_view/one")
7569 This shows the collation sequence to be:
7571 ` ^ _ - , ; : ! ? . ' " ( ) [ ] { } @ * / \ & # % + < = > | ~ $ 0 1 2 3 4 5 6 7 8 9
7573 a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R s S t T u U v V w W x X y Y z Z
7574 Key ranges
7576 Take special care when querying key ranges. For example: the query:
7577 startkey="Abc"&endkey="AbcZZZZ"
7579 will match “ABC” and “abc1”, but not “abc”. This is because UCA sorts
7581 as:
7582 abc < Abc < ABC < abc1 < AbcZZZZZ
7584 For most applications, to avoid problems you should lowercase the
7586 startkey:
7587 startkey="abc"&endkey="abcZZZZZZZZ"
7589 will match all keys starting with [aA][bB][cC]
7591 Complex keys
7593 The query startkey=["foo"]&endkey=["foo",{}] will match most array keys
7594 with “foo” in the first element, such as ["foo","bar"] and
7595 ["foo",["bar","baz"]]. However it will not match
7596 ["foo",{"an":"object"}]
7597 _all_docs
7599 The _all_docs view is a special case because it uses ASCII collation
7600 for doc ids, not UCA:
7601 startkey="_design/"&endkey="_design/ZZZZZZZZ"
7603 will not find _design/abc because ‘Z’ comes before ‘a’ in the ASCII
7605 sequence. A better solution is:
7606 startkey="_design/"&endkey="_design0"
7608 Raw collation
7610 To squeeze a little more performance out of views, you can specify
7611 "options":{"collation":"raw"} within the view definition for native
7612 Erlang collation, especially if you don’t require UCA. This gives a
7613 different collation sequence:
7614 1
7616 false
7617 null
7618 true
7619 {"a":"a"},
7620 ["a"]
7621 "a"
7622 Beware that {} is no longer a suitable “high” key sentinel value. Use a
7624 string like "\ufff0" instead.
7625 Joins With Views
7627 Linked Documents
7628 If your map function emits an object value which has {'_id': XXX} and
7629 you query view with include_docs=true parameter, then CouchDB will
7630 fetch the document with id XXX rather than the document which was pro‐
7631 cessed to emit the key/value pair.
7632 This means that if one document contains the ids of other documents, it
7634 can cause those documents to be fetched in the view too, adjacent to
7635 the same key if required.
7636 For example, if you have the following hierarchically-linked documents:
7638 [
7640 { "_id": "11111" },
7641 { "_id": "22222", "ancestors": ["11111"], "value": "hello" },
7642 { "_id": "33333", "ancestors": ["22222","11111"], "value": "world" }
7643 ]
7644 You can emit the values with the ancestor documents adjacent to them in
7646 the view like this:
7647 function(doc) {
7649 if (doc.value) {
7650 emit([doc.value, 0], null);
7651 if (doc.ancestors) {
7652 for (var i in doc.ancestors) {
7653 emit([doc.value, Number(i)+1], {_id: doc.ancestors[i]});
7654 }
7655 }
7656 }
7657 }
7658 The result you get is:
7660 {
7662 "total_rows": 5,
7663 "offset": 0,
7664 "rows": [
7665 {
7666 "id": "22222",
7667 "key": [
7668 "hello",
7669 0
7670 ],
7671 "value": null,
7672 "doc": {
7673 "_id": "22222",
7674 "_rev": "1-0eee81fecb5aa4f51e285c621271ff02",
7675 "ancestors": [
7676 "11111"
7677 ],
7678 "value": "hello"
7679 }
7680 },
7681 {
7682 "id": "22222",
7683 "key": [
7684 "hello",
7685 1
7686 ],
7687 "value": {
7688 "_id": "11111"
7689 },
7690 "doc": {
7691 "_id": "11111",
7692 "_rev": "1-967a00dff5e02add41819138abb3284d"
7693 }
7694 },
7695 {
7696 "id": "33333",
7697 "key": [
7698 "world",
7699 0
7700 ],
7701 "value": null,
7702 "doc": {
7703 "_id": "33333",
7704 "_rev": "1-11e42b44fdb3d3784602eca7c0332a43",
7705 "ancestors": [
7706 "22222",
7707 "11111"
7708 ],
7709 "value": "world"
7710 }
7711 },
7712 {
7713 "id": "33333",
7714 "key": [
7715 "world",
7716 1
7717 ],
7718 "value": {
7719 "_id": "22222"
7720 },
7721 "doc": {
7722 "_id": "22222",
7723 "_rev": "1-0eee81fecb5aa4f51e285c621271ff02",
7724 "ancestors": [
7725 "11111"
7726 ],
7727 "value": "hello"
7728 }
7729 },
7730 {
7731 "id": "33333",
7732 "key": [
7733 "world",
7734 2
7735 ],
7736 "value": {
7737 "_id": "11111"
7738 },
7739 "doc": {
7740 "_id": "11111",
7741 "_rev": "1-967a00dff5e02add41819138abb3284d"
7742 }
7743 }
7744 ]
7745 }
7746 which makes it very cheap to fetch a document plus all its ancestors in
7748 one query.
7749 Note that the "id" in the row is still that of the originating docu‐
7751 ment. The only difference is that include_docs fetches a different
7752 doc.
7753 The current revision of the document is resolved at query time, not at
7755 the time the view is generated. This means that if a new revision of
7756 the linked document is added later, it will appear in view queries even
7757 though the view itself hasn’t changed. To force a specific revision of
7758 a linked document to be used, emit a "_rev" property as well as "_id".
7759 Using View Collation
7761 Author Christopher Lenz
7762 Date 2007-10-05
7764 Source http://www.cmlenz.net/archives/2007/10/couchdb-joins
7766 Just today, there was a discussion on IRC on how you’d go about model‐
7768 ing a simple blogging system with “post” and “comment” entities, where
7769 any blog post might have N comments. If you’d be using an SQL database,
7770 you’d obviously have two tables with foreign keys and you’d be using
7771 joins. (At least until you needed to add some denormalization).
7772 But what would the “obvious” approach in CouchDB look like?
7774 Approach #1: Comments Inlined
7776 A simple approach would be to have one document per blog post, and
7777 store the comments inside that document:
7778 {
7780 "_id": "myslug",
7781 "_rev": "123456",
7782 "author": "john",
7783 "title": "My blog post",
7784 "content": "Bla bla bla …",
7785 "comments": [
7786 {"author": "jack", "content": "…"},
7787 {"author": "jane", "content": "…"}
7788 ]
7789 }
7790 NOTE:
7792 Of course the model of an actual blogging system would be more
7793 extensive, you’d have tags, timestamps, etc, etc. This is just to
7794 demonstrate the basics.
7795 The obvious advantage of this approach is that the data that belongs
7797 together is stored in one place. Delete the post, and you automatically
7798 delete the corresponding comments, and so on.
7799 You may be thinking that putting the comments inside the blog post doc‐
7801 ument would not allow us to query for the comments themselves, but
7802 you’d be wrong. You could trivially write a CouchDB view that would
7803 return all comments across all blog posts, keyed by author:
7804 function(doc) {
7806 for (var i in doc.comments) {
7807 emit(doc.comments[i].author, doc.comments[i].content);
7808 }
7809 }
7810 Now you could list all comments by a particular user by invoking the
7812 view and passing it a ?key="username" query string parameter.
7813 However, this approach has a drawback that can be quite significant for
7815 many applications: To add a comment to a post, you need to:
7816 · Fetch the blog post document
7818 · Add the new comment to the JSON structure
7820 · Send the updated document to the server
7822 Now if you have multiple client processes adding comments at roughly
7824 the same time, some of them will get a HTTP 409 Conflict error on step
7825 3 (that’s optimistic concurrency in action). For some applications this
7826 makes sense, but in many other apps, you’d want to append new related
7827 data regardless of whether other data has been added in the meantime.
7828 The only way to allow non-conflicting addition of related data is by
7830 putting that related data into separate documents.
7831 Approach #2: Comments Separate
7833 Using this approach you’d have one document per blog post, and one doc‐
7834 ument per comment. The comment documents would have a “backlink” to the
7835 post they belong to.
7836 The blog post document would look similar to the above, minus the com‐
7838 ments property. Also, we’d now have a type property on all our docu‐
7839 ments so that we can tell the difference between posts and comments:
7840 {
7842 "_id": "myslug",
7843 "_rev": "123456",
7844 "type": "post",
7845 "author": "john",
7846 "title": "My blog post",
7847 "content": "Bla bla bla …"
7848 }
7849 The comments themselves are stored in separate documents, which also
7851 have a type property (this time with the value “comment”), and addi‐
7852 tionally feature a post property containing the ID of the post document
7853 they belong to:
7854 {
7856 "_id": "ABCDEF",
7857 "_rev": "123456",
7858 "type": "comment",
7859 "post": "myslug",
7860 "author": "jack",
7861 "content": "…"
7862 }
7863 {
7865 "_id": "DEFABC",
7866 "_rev": "123456",
7867 "type": "comment",
7868 "post": "myslug",
7869 "author": "jane",
7870 "content": "…"
7871 }
7872 To list all comments per blog post, you’d add a simple view, keyed by
7874 blog post ID:
7875 function(doc) {
7877 if (doc.type == "comment") {
7878 emit(doc.post, {author: doc.author, content: doc.content});
7879 }
7880 }
7881 And you’d invoke that view passing it a ?key="post_id" query string
7883 parameter.
7884 Viewing all comments by author is just as easy as before:
7886 function(doc) {
7888 if (doc.type == "comment") {
7889 emit(doc.author, {post: doc.post, content: doc.content});
7890 }
7891 }
7892 So this is better in some ways, but it also has a disadvantage. Imag‐
7894 ine you want to display a blog post with all the associated comments on
7895 the same web page. With our first approach, we needed just a single
7896 request to the CouchDB server, namely a GET request to the document.
7897 With this second approach, we need two requests: a GET request to the
7898 post document, and a GET request to the view that returns all comments
7899 for the post.
7900 That is okay, but not quite satisfactory. Just imagine you wanted to
7902 add threaded comments: you’d now need an additional fetch per comment.
7903 What we’d probably want then would be a way to join the blog post and
7904 the various comments together to be able to retrieve them with a single
7905 HTTP request.
7906 This was when Damien Katz, the author of CouchDB, chimed in to the dis‐
7908 cussion on IRC to show us the way.
7909 Optimization: Using the Power of View Collation
7911 Obvious to Damien, but not at all obvious to the rest of us: it’s
7912 fairly simple to make a view that includes both the content of the blog
7913 post document, and the content of all the comments associated with that
7914 post. The way you do that is by using complex keys. Until now we’ve
7915 been using simple string values for the view keys, but in fact they can
7916 be arbitrary JSON values, so let’s make some use of that:
7917 function(doc) {
7919 if (doc.type == "post") {
7920 emit([doc._id, 0], null);
7921 } else if (doc.type == "comment") {
7922 emit([doc.post, 1], null);
7923 }
7924 }
7925 Okay, this may be confusing at first. Let’s take a step back and look
7927 at what views in CouchDB are really about.
7928 CouchDB views are basically highly efficient on-disk dictionaries that
7930 map keys to values, where the key is automatically indexed and can be
7931 used to filter and/or sort the results you get back from your views.
7932 When you “invoke” a view, you can say that you’re only interested in a
7933 subset of the view rows by specifying a ?key=foo query string parame‐
7934 ter. Or you can specify ?startkey=foo and/or ?endkey=bar query string
7935 parameters to fetch rows over a range of keys. Finally, by adding
7936 ?include_docs=true to the query, the result will include the full body
7937 of each emitted document.
7938 It’s also important to note that keys are always used for collating
7940 (i.e. sorting) the rows. CouchDB has well defined (but as of yet
7941 undocumented) rules for comparing arbitrary JSON objects for collation.
7942 For example, the JSON value ["foo", 2] is sorted after (considered
7943 “greater than”) the values ["foo"] or ["foo", 1, "bar"], but before
7944 e.g. ["foo", 2, "bar"]. This feature enables a whole class of tricks
7945 that are rather non-obvious…
7946 SEE ALSO:
7948 views/collation
7949 With that in mind, let’s return to the view function above. First note
7951 that, unlike the previous view functions we’ve used here, this view
7952 handles both “post” and “comment” documents, and both of them end up as
7953 rows in the same view. Also, the key in this view is not just a simple
7954 string, but an array. The first element in that array is always the ID
7955 of the post, regardless of whether we’re processing an actual post doc‐
7956 ument, or a comment associated with a post. The second element is 0 for
7957 post documents, and 1 for comment documents.
7958 Let’s assume we have two blog posts in our database. Without limiting
7960 the view results via key, startkey, or endkey, we’d get back something
7961 like the following:
7962 {
7964 "total_rows": 5, "offset": 0, "rows": [{
7965 "id": "myslug",
7966 "key": ["myslug", 0],
7967 "value": null
7968 }, {
7969 "id": "ABCDEF",
7970 "key": ["myslug", 1],
7971 "value": null
7972 }, {
7973 "id": "DEFABC",
7974 "key": ["myslug", 1],
7975 "value": null
7976 }, {
7977 "id": "other_slug",
7978 "key": ["other_slug", 0],
7979 "value": null
7980 }, {
7981 "id": "CDEFAB",
7982 "key": ["other_slug", 1],
7983 "value": null
7984 },
7985 ]
7986 }
7987 NOTE:
7989 The ... placeholders here would contain the complete JSON encoding
7990 of the corresponding documents
7991 Now, to get a specific blog post and all associated comments, we’d
7993 invoke that view with the query string:
7994 ?startkey=["myslug"]&endkey=["myslug", 2]&include_docs=true
7996 We’d get back the first three rows, those that belong to the myslug
7998 post, but not the others, along with the full bodies of each document.
7999 Et voila, we now have the data we need to display a post with all asso‐
8000 ciated comments, retrieved via a single GET request.
8001 You may be asking what the 0 and 1 parts of the keys are for. They’re
8003 simply to ensure that the post document is always sorted before the the
8004 associated comment documents. So when you get back the results from
8005 this view for a specific post, you’ll know that the first row contains
8006 the data for the blog post itself, and the remaining rows contain the
8007 comment data.
8008 One remaining problem with this model is that comments are not ordered,
8010 but that’s simply because we don’t have date/time information associ‐
8011 ated with them. If we had, we’d add the timestamp as third element of
8012 the key array, probably as ISO date/time strings. Now we would continue
8013 using the query string ?startkey=["myslug"]&endkey=["myslug",
8014 2]&include_docs=true to fetch the blog post and all associated com‐
8015 ments, only now they’d be in chronological order.
8016 View Cookbook for SQL Jockeys
8018 This is a collection of some common SQL queries and how to get the same
8019 result in CouchDB. The key to remember here is that CouchDB does not
8020 work like an SQL database at all, and that best practices from the SQL
8021 world do not translate well or at all to CouchDB. This document’s
8022 “cookbook” assumes that you are familiar with the CouchDB basics such
8023 as creating and updating databases and documents.
8024 Using Views
8026 How you would do this in SQL:
8027 CREATE TABLE
8029 or:
8031 ALTER TABLE
8033 How you can do this in CouchDB?
8035 Using views is a two-step process. First you define a view; then you
8037 query it. This is analogous to defining a table structure (with
8038 indexes) using CREATE TABLE or ALTER TABLE and querying it using an SQL
8039 query.
8040 Defining a View
8042 Defining a view is done by creating a special document in a CouchDB
8043 database. The only real specialness is the _id of the document, which
8044 starts with _design/ — for example, _design/application. Other than
8045 that, it is just a regular CouchDB document. To make sure CouchDB
8046 understands that you are defining a view, you need to prepare the con‐
8047 tents of that design document in a special format. Here is an example:
8048 {
8050 "_id": "_design/application",
8051 "_rev": "1-C1687D17",
8052 "views": {
8053 "viewname": {
8054 "map": "function(doc) { ... }",
8055 "reduce": "function(keys, values) { ... }"
8056 }
8057 }
8058 }
8059 We are defining a view viewname. The definition of the view consists of
8061 two functions: the map function and the reduce function. Specifying a
8062 reduce function is optional. We’ll look at the nature of the functions
8063 later. Note that viewname can be whatever you like: users, by-name, or
8064 by-date are just some examples.
8065 A single design document can also include multiple view definitions,
8067 each identified by a unique name:
8068 {
8070 "_id": "_design/application",
8071 "_rev": "1-C1687D17",
8072 "views": {
8073 "viewname": {
8074 "map": "function(doc) { ... }",
8075 "reduce": "function(keys, values) { ... }"
8076 },
8077 "anotherview": {
8078 "map": "function(doc) { ... }",
8079 "reduce": "function(keys, values) { ... }"
8080 }
8081 }
8082 }
8083 Querying a View
8085 The name of the design document and the name of the view are signifi‐
8086 cant for querying the view. To query the view viewname, you perform an
8087 HTTP GET request to the following URI:
8088 /database/_design/application/_view/viewname
8090 database is the name of the database you created your design document
8092 in. Next up is the design document name, and then the view name pre‐
8093 fixed with _view/. To query anotherview, replace viewname in that URI
8094 with anotherview. If you want to query a view in a different design
8095 document, adjust the design document name.
8096 MapReduce Functions
8098 MapReduce is a concept that solves problems by applying a two-step
8099 process, aptly named the map phase and the reduce phase. The map phase
8100 looks at all documents in CouchDB separately one after the other and
8101 creates a map result. The map result is an ordered list of key/value
8102 pairs. Both key and value can be specified by the user writing the map
8103 function. A map function may call the built-in emit(key, value) func‐
8104 tion 0 to N times per document, creating a row in the map result per
8105 invocation.
8106 CouchDB is smart enough to run a map function only once for every docu‐
8108 ment, even on subsequent queries on a view. Only changes to documents
8109 or new documents need to be processed anew.
8110 Map functions
8112 Map functions run in isolation for every document. They can’t modify
8113 the document, and they can’t talk to the outside world—they can’t have
8114 side effects. This is required so that CouchDB can guarantee correct
8115 results without having to recalculate a complete result when only one
8116 document gets changed.
8117 The map result looks like this:
8119 {"total_rows":3,"offset":0,"rows":[
8121 {"id":"fc2636bf50556346f1ce46b4bc01fe30","key":"Lena","value":5},
8122 {"id":"1fb2449f9b9d4e466dbfa47ebe675063","key":"Lisa","value":4},
8123 {"id":"8ede09f6f6aeb35d948485624b28f149","key":"Sarah","value":6}
8124 ]}
8125 It is a list of rows sorted by the value of key. The id is added auto‐
8127 matically and refers back to the document that created this row. The
8128 value is the data you’re looking for. For example purposes, it’s the
8129 girl’s age.
8130 The map function that produces this result is:
8132 function(doc) {
8134 if(doc.name && doc.age) {
8135 emit(doc.name, doc.age);
8136 }
8137 }
8138 It includes the if statement as a sanity check to ensure that we’re
8140 operating on the right fields and calls the emit function with the name
8141 and age as the key and value.
8142 Look Up by Key
8144 How you would do this in SQL:
8145 SELECT field FROM table WHERE value="searchterm"
8147 How you can do this in CouchDB?
8149 Use case: get a result (which can be a record or set of records) asso‐
8151 ciated with a key (“searchterm”).
8152 To look something up quickly, regardless of the storage mechanism, an
8154 index is needed. An index is a data structure optimized for quick
8155 search and retrieval. CouchDB’s map result is stored in such an index,
8156 which happens to be a B+ tree.
8157 To look up a value by “searchterm”, we need to put all values into the
8159 key of a view. All we need is a simple map function:
8160 function(doc) {
8162 if(doc.value) {
8163 emit(doc.value, null);
8164 }
8165 }
8166 This creates a list of documents that have a value field sorted by the
8168 data in the value field. To find all the records that match
8169 “searchterm”, we query the view and specify the search term as a query
8170 parameter:
8171 /database/_design/application/_view/viewname?key="searchterm"
8173 Consider the documents from the previous section, and say we’re index‐
8175 ing on the age field of the documents to find all the five-year-olds:
8176 function(doc) {
8178 if(doc.age && doc.name) {
8179 emit(doc.age, doc.name);
8180 }
8181 }
8182 Query:
8184 /ladies/_design/ladies/_view/age?key=5
8186 Result:
8188 {"total_rows":3,"offset":1,"rows":[
8190 {"id":"fc2636bf50556346f1ce46b4bc01fe30","key":5,"value":"Lena"}
8191 ]}
8192 Easy.
8194 Note that you have to emit a value. The view result includes the asso‐
8196 ciated document ID in every row. We can use it to look up more data
8197 from the document itself. We can also use the ?include_docs=true param‐
8198 eter to have CouchDB fetch the individual documents for us.
8199 Look Up by Prefix
8201 How you would do this in SQL:
8202 SELECT field FROM table WHERE value LIKE "searchterm%"
8204 How you can do this in CouchDB?
8206 Use case: find all documents that have a field value that starts with
8208 searchterm. For example, say you stored a MIME type (like text/html or
8209 image/jpg) for each document and now you want to find all documents
8210 that are images according to the MIME type.
8211 The solution is very similar to the previous example: all we need is a
8213 map function that is a little more clever than the first one. But
8214 first, an example document:
8215 {
8217 "_id": "Hugh Laurie",
8218 "_rev": "1-9fded7deef52ac373119d05435581edf",
8219 "mime-type": "image/jpg",
8220 "description": "some dude"
8221 }
8222 The clue lies in extracting the prefix that we want to search for from
8224 our document and putting it into our view index. We use a regular
8225 expression to match our prefix:
8226 function(doc) {
8228 if(doc["mime-type"]) {
8229 // from the start (^) match everything that is not a slash ([^\/]+) until
8230 // we find a slash (\/). Slashes needs to be escaped with a backslash (\/)
8231 var prefix = doc["mime-type"].match(/^[^\/]+\//);
8232 if(prefix) {
8233 emit(prefix, null);
8234 }
8235 }
8236 }
8237 We can now query this view with our desired MIME type prefix and not
8239 only find all images, but also text, video, and all other formats:
8240 /files/_design/finder/_view/by-mime-type?key="image/"
8242 Aggregate Functions
8244 How you would do this in SQL:
8245 SELECT COUNT(field) FROM table
8247 How you can do this in CouchDB?
8249 Use case: calculate a derived value from your data.
8251 We haven’t explained reduce functions yet. Reduce functions are similar
8253 to aggregate functions in SQL. They compute a value over multiple docu‐
8254 ments.
8255 To explain the mechanics of reduce functions, we’ll create one that
8257 doesn’t make a whole lot of sense. But this example is easy to under‐
8258 stand. We’ll explore more useful reductions later.
8259 Reduce functions operate on the output of the map function (also called
8261 the map result or intermediate result). The reduce function’s job,
8262 unsurprisingly, is to reduce the list that the map function produces.
8263 Here’s what our summing reduce function looks like:
8265 function(keys, values) {
8267 var sum = 0;
8268 for(var idx in values) {
8269 sum = sum + values[idx];
8270 }
8271 return sum;
8272 }
8273 Here’s an alternate, more idiomatic JavaScript version:
8275 function(keys, values) {
8277 var sum = 0;
8278 values.forEach(function(element) {
8279 sum = sum + element;
8280 });
8281 return sum;
8282 }
8283 NOTE:
8285 Don’t miss effective built-in reduce functions like _sum and _count
8286 This reduce function takes two arguments: a list of keys and a list of
8288 values. For our summing purposes we can ignore the keys-list and con‐
8289 sider only the value list. We’re looping over the list and add each
8290 item to a running total that we’re returning at the end of the func‐
8291 tion.
8292 You’ll see one difference between the map and the reduce function. The
8294 map function uses emit() to create its result, whereas the reduce func‐
8295 tion returns a value.
8296 For example, from a list of integer values that specify the age, calcu‐
8298 late the sum of all years of life for the news headline, “786 life
8299 years present at event.” A little contrived, but very simple and thus
8300 good for demonstration purposes. Consider the documents and the map
8301 view we used earlier in this document.
8302 The reduce function to calculate the total age of all girls is:
8304 function(keys, values) {
8306 return sum(values);
8307 }
8308 Note that, instead of the two earlier versions, we use CouchDB’s prede‐
8310 fined sum() function. It does the same thing as the other two, but it
8311 is such a common piece of code that CouchDB has it included.
8312 The result for our reduce view now looks like this:
8314 {"rows":[
8316 {"key":null,"value":15}
8317 ]}
8318 The total sum of all age fields in all our documents is 15. Just what
8320 we wanted. The key member of the result object is null, as we can’t
8321 know anymore which documents took part in the creation of the reduced
8322 result. We’ll cover more advanced reduce cases later on.
8323 As a rule of thumb, the reduce function should reduce to a single
8325 scalar value. That is, an integer; a string; or a small, fixed-size
8326 list or object that includes an aggregated value (or values) from the
8327 values argument. It should never just return values or similar.
8328 CouchDB will give you a warning if you try to use reduce “the wrong
8329 way”:
8330 {
8332 "error":"reduce_overflow_error",
8333 "message":"Reduce output must shrink more rapidly: Current output: ..."
8334 }
8335 Get Unique Values
8337 How you would do this in SQL:
8338 SELECT DISTINCT field FROM table
8340 How you can do this in CouchDB?
8342 Getting unique values is not as easy as adding a keyword. But a reduce
8344 view and a special query parameter give us the same result. Let’s say
8345 you want a list of tags that your users have tagged themselves with and
8346 no duplicates.
8347 First, let’s look at the source documents. We punt on _id and _rev
8349 attributes here:
8350 {
8352 "name":"Chris",
8353 "tags":["mustache", "music", "couchdb"]
8354 }
8355 {
8357 "name":"Noah",
8358 "tags":["hypertext", "philosophy", "couchdb"]
8359 }
8360 {
8362 "name":"Jan",
8363 "tags":["drums", "bike", "couchdb"]
8364 }
8365 Next, we need a list of all tags. A map function will do the trick:
8367 function(doc) {
8369 if(doc.name && doc.tags) {
8370 doc.tags.forEach(function(tag) {
8371 emit(tag, null);
8372 });
8373 }
8374 }
8375 The result will look like this:
8377 {"total_rows":9,"offset":0,"rows":[
8379 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"bike","value":null},
8380 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"couchdb","value":null},
8381 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"couchdb","value":null},
8382 {"id":"da5ea89448a4506925823f4d985aabbd","key":"couchdb","value":null},
8383 {"id":"3525ab874bc4965fa3cda7c549e92d30","key":"drums","value":null},
8384 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"hypertext","value":null},
8385 {"id":"da5ea89448a4506925823f4d985aabbd","key":"music","value":null},
8386 {"id":"da5ea89448a4506925823f4d985aabbd","key":"mustache","value":null},
8387 {"id":"53f82b1f0ff49a08ac79a9dff41d7860","key":"philosophy","value":null}
8388 ]}
8389 As promised, these are all the tags, including duplicates. Since each
8391 document gets run through the map function in isolation, it cannot know
8392 if the same key has been emitted already. At this stage, we need to
8393 live with that. To achieve uniqueness, we need a reduce:
8394 function(keys, values) {
8396 return true;
8397 }
8398 This reduce doesn’t do anything, but it allows us to specify a special
8400 query parameter when querying the view:
8401 /dudes/_design/dude-data/_view/tags?group=true
8403 CouchDB replies:
8405 {"rows":[
8407 {"key":"bike","value":true},
8408 {"key":"couchdb","value":true},
8409 {"key":"drums","value":true},
8410 {"key":"hypertext","value":true},
8411 {"key":"music","value":true},
8412 {"key":"mustache","value":true},
8413 {"key":"philosophy","value":true}
8414 ]}
8415 In this case, we can ignore the value part because it is always true,
8417 but the result includes a list of all our tags and no duplicates!
8418 With a small change we can put the reduce to good use, too. Let’s see
8420 how many of the non-unique tags are there for each tag. To calculate
8421 the tag frequency, we just use the summing up we already learned about.
8422 In the map function, we emit a 1 instead of null:
8423 function(doc) {
8425 if(doc.name && doc.tags) {
8426 doc.tags.forEach(function(tag) {
8427 emit(tag, 1);
8428 });
8429 }
8430 }
8431 In the reduce function, we return the sum of all values:
8433 function(keys, values) {
8435 return sum(values);
8436 }
8437 Now, if we query the view with the ?group=true parameter, we get back
8439 the count for each tag:
8440 {"rows":[
8442 {"key":"bike","value":1},
8443 {"key":"couchdb","value":3},
8444 {"key":"drums","value":1},
8445 {"key":"hypertext","value":1},
8446 {"key":"music","value":1},
8447 {"key":"mustache","value":1},
8448 {"key":"philosophy","value":1}
8449 ]}
8450 Enforcing Uniqueness
8452 How you would do this in SQL:
8453 UNIQUE KEY(column)
8455 How you can do this in CouchDB?
8457 Use case: your applications require that a certain value exists only
8459 once in a database.
8460 This is an easy one: within a CouchDB database, each document must have
8462 a unique _id field. If you require unique values in a database, just
8463 assign them to a document’s _id field and CouchDB will enforce unique‐
8464 ness for you.
8465 There’s one caveat, though: in the distributed case, when you are run‐
8467 ning more than one CouchDB node that accepts write requests, uniqueness
8468 can be guaranteed only per node or outside of CouchDB. CouchDB will
8469 allow two identical IDs to be written to two different nodes. On repli‐
8470 cation, CouchDB will detect a conflict and flag the document accord‐
8471 ingly.
8472 Pagination Recipe
8474 This recipe explains how to paginate over view results. Pagination is
8475 a user interface (UI) pattern that allows the display of a large number
8476 of rows (the result set) without loading all the rows into the UI at
8477 once. A fixed-size subset, the page, is displayed along with next and
8478 previous links or buttons that can move the viewport over the result
8479 set to an adjacent page.
8480 We assume you’re familiar with creating and querying documents and
8482 views as well as the multiple view query options.
8483 Example Data
8485 To have some data to work with, we’ll create a list of bands, one docu‐
8486 ment per band:
8487 { "name":"Biffy Clyro" }
8489 { "name":"Foo Fighters" }
8491 { "name":"Tool" }
8493 { "name":"Nirvana" }
8495 { "name":"Helmet" }
8497 { "name":"Tenacious D" }
8499 { "name":"Future of the Left" }
8501 { "name":"A Perfect Circle" }
8503 { "name":"Silverchair" }
8505 { "name":"Queens of the Stone Age" }
8507 { "name":"Kerub" }
8509 A View
8511 We need a simple map function that gives us an alphabetical list of
8512 band names. This should be easy, but we’re adding extra smarts to fil‐
8513 ter out “The” and “A” in front of band names to put them into the right
8514 position:
8515 function(doc) {
8517 if(doc.name) {
8518 var name = doc.name.replace(/^(A|The) /, "");
8519 emit(name, null);
8520 }
8521 }
8522 The views result is an alphabetical list of band names. Now say we want
8524 to display band names five at a time and have a link pointing to the
8525 next five names that make up one page, and a link for the previous
8526 five, if we’re not on the first page.
8527 We learned how to use the startkey, limit, and skip parameters in ear‐
8529 lier documents. We’ll use these again here. First, let’s have a look at
8530 the full result set:
8531 {"total_rows":11,"offset":0,"rows":[
8533 {"id":"a0746072bba60a62b01209f467ca4fe2","key":"Biffy Clyro","value":null},
8534 {"id":"b47d82284969f10cd1b6ea460ad62d00","key":"Foo Fighters","value":null},
8535 {"id":"45ccde324611f86ad4932555dea7fce0","key":"Tenacious D","value":null},
8536 {"id":"d7ab24bb3489a9010c7d1a2087a4a9e4","key":"Future of the Left","value":null},
8537 {"id":"ad2f85ef87f5a9a65db5b3a75a03cd82","key":"Helmet","value":null},
8538 {"id":"a2f31cfa68118a6ae9d35444fcb1a3cf","key":"Nirvana","value":null},
8539 {"id":"67373171d0f626b811bdc34e92e77901","key":"Kerub","value":null},
8540 {"id":"3e1b84630c384f6aef1a5c50a81e4a34","key":"Perfect Circle","value":null},
8541 {"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"Queens of the Stone Age","value":null},
8542 {"id":"dcdaf08242a4be7da1a36e25f4f0b022","key":"Silverchair","value":null},
8543 {"id":"fd590d4ad53771db47b0406054f02243","key":"Tool","value":null}
8544 ]}
8545 Setup
8547 The mechanics of paging are very simple:
8548 · Display first page
8550 · If there are more rows to show, show next link
8552 · Draw subsequent page
8554 · If this is not the first page, show a previous link
8556 · If there are more rows to show, show next link
8558 Or in a pseudo-JavaScript snippet:
8560 var result = new Result();
8562 var page = result.getPage();
8563 page.display();
8565 if(result.hasPrev()) {
8567 page.display_link('prev');
8568 }
8569 if(result.hasNext()) {
8571 page.display_link('next');
8572 }
8573 Paging
8575 To get the first five rows from the view result, you use the ?limit=5
8576 query parameter:
8577 curl -X GET http://127.0.0.1:5984/artists/_design/artists/_view/by-name?limit=5
8579 The result:
8581 {"total_rows":11,"offset":0,"rows":[
8583 {"id":"a0746072bba60a62b01209f467ca4fe2","key":"Biffy Clyro","value":null},
8584 {"id":"b47d82284969f10cd1b6ea460ad62d00","key":"Foo Fighters","value":null},
8585 {"id":"45ccde324611f86ad4932555dea7fce0","key":"Tenacious D","value":null},
8586 {"id":"d7ab24bb3489a9010c7d1a2087a4a9e4","key":"Future of the Left","value":null},
8587 {"id":"ad2f85ef87f5a9a65db5b3a75a03cd82","key":"Helmet","value":null}
8588 ]}
8589 By comparing the total_rows value to our limit value, we can determine
8591 if there are more pages to display. We also know by the offset member
8592 that we are on the first page. We can calculate the value for skip= to
8593 get the results for the next page:
8594 var rows_per_page = 5;
8596 var page = (offset / rows_per_page) + 1; // == 1
8597 var skip = page * rows_per_page; // == 5 for the first page, 10 for the second ...
8598 So we query CouchDB with:
8600 curl -X GET 'http://127.0.0.1:5984/artists/_design/artists/_view/by-name?limit=5&skip=5'
8602 Note we have to use ' (single quotes) to escape the & character that is
8604 special to the shell we execute curl in.
8605 The result:
8607 {"total_rows":11,"offset":5,"rows":[
8609 {"id":"a2f31cfa68118a6ae9d35444fcb1a3cf","key":"Nirvana","value":null},
8610 {"id":"67373171d0f626b811bdc34e92e77901","key":"Kerub","value":null},
8611 {"id":"3e1b84630c384f6aef1a5c50a81e4a34","key":"Perfect Circle","value":null},
8612 {"id":"84a371a7b8414237fad1b6aaf68cd16a","key":"Queens of the Stone Age",
8613 "value":null},
8614 {"id":"dcdaf08242a4be7da1a36e25f4f0b022","key":"Silverchair","value":null}
8615 ]}
8616 Implementing the hasPrev() and hasNext() method is pretty straightfor‐
8618 ward:
8619 function hasPrev()
8621 {
8622 return page > 1;
8623 }
8624 function hasNext()
8626 {
8627 var last_page = Math.floor(total_rows / rows_per_page) +
8628 (total_rows % rows_per_page);
8629 return page != last_page;
8630 }
8631 Paging (Alternate Method)
8633 The method described above performed poorly with large skip values
8634 until CouchDB 1.2. Additionally, some use cases may call for the fol‐
8635 lowing alternate method even with newer versions of CouchDB. One such
8636 case is when duplicate results should be prevented. Using skip alone it
8637 is possible for new documents to be inserted during pagination which
8638 could change the offset of the start of the subsequent page.
8639 A correct solution is not much harder. Instead of slicing the result
8641 set into equally sized pages, we look at 10 rows at a time and use
8642 startkey to jump to the next 10 rows. We even use skip, but only with
8643 the value 1.
8644 Here is how it works:
8646 · Request rows_per_page + 1 rows from the view
8648 · Display rows_per_page rows, store + 1 row as next_startkey and
8650 next_startkey_docid
8651 · As page information, keep startkey and next_startkey
8653 · Use the next_* values to create the next link, and use the others to
8655 create the previous link
8656 The trick to finding the next page is pretty simple. Instead of
8658 requesting 10 rows for a page, you request 11 rows, but display only 10
8659 and use the values in the 11th row as the startkey for the next page.
8660 Populating the link to the previous page is as simple as carrying the
8661 current startkey over to the next page. If there’s no previous
8662 startkey, we are on the first page. We stop displaying the link to the
8663 next page if we get rows_per_page or less rows back. This is called
8664 linked list pagination, as we go from page to page, or list item to
8665 list item, instead of jumping directly to a pre-computed page. There is
8666 one caveat, though. Can you spot it?
8667 CouchDB view keys do not have to be unique; you can have multiple index
8669 entries read. What if you have more index entries for a key than rows
8670 that should be on a page? startkey jumps to the first row, and you’d be
8671 screwed if CouchDB didn’t have an additional parameter for you to use.
8672 All view keys with the same value are internally sorted by docid, that
8673 is, the ID of the document that created that view row. You can use the
8674 startkey_docid and endkey_docid parameters to get subsets of these
8675 rows. For pagination, we still don’t need endkey_docid, but
8676 startkey_docid is very handy. In addition to startkey and limit, you
8677 also use startkey_docid for pagination if, and only if, the extra row
8678 you fetch to find the next page has the same key as the current
8679 startkey.
8680 It is important to note that the *_docid parameters only work in addi‐
8682 tion to the *key parameters and are only useful to further narrow down
8683 the result set of a view for a single key. They do not work on their
8684 own (the one exception being the built-in _all_docs view that already
8685 sorts by document ID).
8686 The advantage of this approach is that all the key operations can be
8688 performed on the super-fast B-tree index behind the view. Looking up a
8689 page doesn’t include scanning through hundreds and thousands of rows
8690 unnecessarily.
8691 Jump to Page
8693 One drawback of the linked list style pagination is that you can’t
8694 pre-compute the rows for a particular page from the page number and the
8695 rows per page. Jumping to a specific page doesn’t really work. Our gut
8696 reaction, if that concern is raised, is, “Not even Google is doing
8697 that!” and we tend to get away with it. Google always pretends on the
8698 first page to find 10 more pages of results. Only if you click on the
8699 second page (something very few people actually do) might Google dis‐
8700 play a reduced set of pages. If you page through the results, you get
8701 links for the previous and next 10 pages, but no more. Pre-computing
8702 the necessary startkey and startkey_docid for 20 pages is a feasible
8703 operation and a pragmatic optimization to know the rows for every page
8704 in a result set that is potentially tens of thousands of rows long, or
8705 more.
8706 If you really do need to jump to a page over the full range of docu‐
8708 ments (we have seen applications that require that), you can still
8709 maintain an integer value index as the view index and take a hybrid
8710 approach at solving pagination.
8711 Search
8713 Search indexes enable you to query a database by using the Lucene Query
8714 Parser Syntax. A search index uses one, or multiple, fields from your
8715 documents. You can use a search index to run queries, find documents
8716 based on the content they contain, or work with groups, facets, or geo‐
8717 graphical searches.
8718 WARNING:
8720 Search cannot function unless it has a functioning, cluster-con‐
8721 nected Clouseau instance. See Search Plugin Installation for
8722 details.
8723 To create a search index, you add a JavaScript function to a design
8725 document in the database. An index builds after processing one search
8726 request or after the server detects a document update. The index func‐
8727 tion takes the following parameters:
8728 1. Field name - The name of the field you want to use when you query
8730 the index. If you set this parameter to default, then this field is
8731 queried if no field is specified in the query syntax.
8732 2. Data that you want to index, for example, doc.address.country.
8734 3. (Optional) The third parameter includes the following fields:
8736 boost, facet, index, and store. These fields are described in more
8737 detail later.
8738 By default, a search index response returns 25 rows. The number of rows
8740 that is returned can be changed by using the limit parameter. Each
8741 response includes a bookmark field. You can include the value of the
8742 bookmark field in later queries to look through the responses.
8743 Example design document that defines a search index:
8745 {
8747 "_id": "_design/search_example",
8748 "indexes": {
8749 "animals": {
8750 "index": "function(doc){ ... }"
8751 }
8752 }
8753 }
8754 A search index will inherit the partitioning type from the options.par‐
8756 titioned field of the design document that contains it.
8757 Index functions
8759 Attempting to index by using a data field that does not exist fails. To
8760 avoid this problem, use the appropriate guard clause.
8761 NOTE:
8763 Your indexing functions operate in a memory-constrained environment
8764 where the document itself forms a part of the memory that is used in
8765 that environment. Your code’s stack and document must fit inside
8766 this memory. In other words, a document must be loaded in order to
8767 be indexed. Documents are limited to a maximum size of 64 MB.
8768 NOTE:
8770 Within a search index, do not index the same field name with more
8771 than one data type. If the same field name is indexed with different
8772 data types in the same search index function, you might get an error
8773 when querying the search index that says the field “was indexed
8774 without position data.” For example, do not include both of these
8775 lines in the same search index function, as they index the myfield
8776 field as two different data types: a string "this is a string" and a
8777 number 123.
8778 index("myfield", "this is a string");
8780 index("myfield", 123);
8781 The function that is contained in the index field is a JavaScript func‐
8783 tion that is called for each document in the database. The function
8784 takes the document as a parameter, extracts some data from it, and then
8785 calls the function that is defined in the index field to index that
8786 data.
8787 The index function takes three parameters, where the third parameter is
8789 optional.
8790 The first parameter is the name of the field you intend to use when
8792 querying the index, and which is specified in the Lucene syntax portion
8793 of subsequent queries. An example appears in the following query:
8794 query=color:red
8796 The Lucene field name color is the first parameter of the index func‐
8798 tion.
8799 The query parameter can be abbreviated to q, so another way of writing
8801 the query is as follows:
8802 q=color:red
8804 If the special value "default" is used when you define the name, you do
8806 not have to specify a field name at query time. The effect is that the
8807 query can be simplified:
8808 query=red
8810 The second parameter is the data to be indexed. Keep the following
8812 information in mind when you index your data:
8813 · This data must be only a string, number, or boolean. Other types will
8815 cause an error to be thrown by the index function call.
8816 · If an error is thrown when running your function, for this reason or
8818 others, the document will not be added to that search index.
8819 The third, optional, parameter is a JavaScript object with the follow‐
8821 ing fields:
8822 Index function (optional parameter)
8824 · boost - A number that specifies the relevance in search results. Con‐
8826 tent that is indexed with a boost value greater than 1 is more rele‐
8827 vant than content that is indexed without a boost value. Content with
8828 a boost value less than one is not so relevant. Value is a positive
8829 floating point number. Default is 1 (no boosting).
8830 · facet - Creates a faceted index. See Faceting. Values are true or
8832 false. Default is false.
8833 · index - Whether the data is indexed, and if so, how. If set to false,
8835 the data cannot be used for searches, but can still be retrieved from
8836 the index if store is set to true. See Analyzers. Values are true or
8837 false. Default is true
8838 · store - If true, the value is returned in the search result; other‐
8840 wise, the value is not returned. Values are true or false. Default is
8841 false.
8842 NOTE:
8844 If you do not set the store parameter, the index data results for
8845 the document are not returned in response to a query.
8846 Example search index function:
8848 function(doc) {
8850 index("default", doc._id);
8851 if (doc.min_length) {
8852 index("min_length", doc.min_length, {"store": true});
8853 }
8854 if (doc.diet) {
8855 index("diet", doc.diet, {"store": true});
8856 }
8857 if (doc.latin_name) {
8858 index("latin_name", doc.latin_name, {"store": true});
8859 }
8860 if (doc.class) {
8861 index("class", doc.class, {"store": true});
8862 }
8863 }
8864 Index guard clauses
8866 The index function requires the name of the data field to index as the
8867 second parameter. However, if that data field does not exist for the
8868 document, an error occurs. The solution is to use an appropriate
8869 ‘guard clause’ that checks if the field exists, and contains the
8870 expected type of data, before any attempt to create the corresponding
8871 index.
8872 Example of failing to check whether the index data field exists:
8874 if (doc.min_length) {
8876 index("min_length", doc.min_length, {"store": true});
8877 }
8878 You might use the JavaScript typeof function to implement the guard
8880 clause test. If the field exists and has the expected type, the correct
8881 type name is returned, so the guard clause test succeeds and it is safe
8882 to use the index function. If the field does not exist, you would not
8883 get back the expected type of the field, therefore you would not
8884 attempt to index the field.
8885 JavaScript considers a result to be false if one of the following val‐
8887 ues is tested:
8888 · ‘undefined’
8890 · null
8892 · The number +0
8894 · The number -0
8896 · NaN (not a number)
8898 · “” (the empty string)
8900 Using a guard clause to check whether the required data field exists,
8902 and holds a number, before an attempt to index:
8903 if (typeof(doc.min_length) === 'number') {
8905 index("min_length", doc.min_length, {"store": true});
8906 }
8907 Use a generic guard clause test to ensure that the type of the candi‐
8909 date data field is defined.
8910 Example of a ‘generic’ guard clause:
8912 if (typeof(doc.min_length) !== 'undefined') {
8914 // The field exists, and does have a type, so we can proceed to index using it.
8915 ...
8916 }
8917 Analyzers
8919 Analyzers are settings that define how to recognize terms within text.
8920 Analyzers can be helpful if you need to index multiple languages.
8921 Here’s the list of generic analyzers, and their descriptions, that are
8923 supported by search:
8924 · classic - The standard Lucene analyzer, circa release 3.1.
8926 · email - Like the standard analyzer, but tries harder to match an
8928 email address as a complete token.
8929 · keyword - Input is not tokenized at all.
8931 · simple - Divides text at non-letters.
8933 · standard - The default analyzer. It implements the Word Break rules
8935 from the Unicode Text Segmentation algorithm
8936 · whitespace - Divides text at white space boundaries.
8938 Example analyzer document:
8940 {
8942 "_id": "_design/analyzer_example",
8943 "indexes": {
8944 "INDEX_NAME": {
8945 "index": "function (doc) { ... }",
8946 "analyzer": "$ANALYZER_NAME"
8947 }
8948 }
8949 }
8950 Language-specific analyzers
8952 These analyzers omit common words in the specific language, and many
8953 also remove prefixes and suffixes. The name of the language is also
8954 the name of the analyzer. See package org.apache.lucene.analysis for
8955 more information.
8956 ┌───────────┬───────────────────────────┐
8958 │Language │ Analyzer │
8959 ├───────────┼───────────────────────────┤
8960 │arabic │ org.apache.lucene.analy‐ │
8961 │ │ sis.ar.ArabicAnalyzer │
8962 ├───────────┼───────────────────────────┤
8963 │armenian │ org.apache.lucene.analy‐ │
8964 │ │ sis.hy.ArmenianAnalyzer │
8965 ├───────────┼───────────────────────────┤
8966 │basque │ org.apache.lucene.analy‐ │
8967 │ │ sis.eu.BasqueAnalyzer │
8968 ├───────────┼───────────────────────────┤
8969 │bulgarian │ org.apache.lucene.analy‐ │
8970 │ │ sis.bg.BulgarianAnalyzer │
8971 ├───────────┼───────────────────────────┤
8972 │brazilian │ org.apache.lucene.analy‐ │
8973 │ │ sis.br.BrazilianAnalyzer │
8974 ├───────────┼───────────────────────────┤
8975 │catalan │ org.apache.lucene.analy‐ │
8976 │ │ sis.ca.CatalanAnalyzer │
8977 ├───────────┼───────────────────────────┤
8978 │cjk │ org.apache.lucene.analy‐ │
8979 │ │ sis.cjk.CJKAnalyzer │
8980 ├───────────┼───────────────────────────┤
8981 │chinese │ org.apache.lucene.analy‐ │
8982 │ │ sis.cn.smart.SmartChine‐ │
8983 │ │ seAnalyzer │
8984 ├───────────┼───────────────────────────┤
8985 │czech │ org.apache.lucene.analy‐ │
8986 │ │ sis.cz.CzechAnalyzer │
8987 ├───────────┼───────────────────────────┤
8988 │danish │ org.apache.lucene.analy‐ │
8989 │ │ sis.da.DanishAnalyzer │
8990 ├───────────┼───────────────────────────┤
8991 │dutch │ org.apache.lucene.analy‐ │
8992 │ │ sis.nl.DutchAnalyzer │
8993 ├───────────┼───────────────────────────┤
8994 │english │ org.apache.lucene.analy‐ │
8995 │ │ sis.en.EnglishAnalyzer │
8996 ├───────────┼───────────────────────────┤
8997 │finnish │ org.apache.lucene.analy‐ │
8998 │ │ sis.fi.FinnishAnalyzer │
8999 ├───────────┼───────────────────────────┤
9000 │french │ org.apache.lucene.analy‐ │
9001 │ │ sis.fr.FrenchAnalyzer │
9002 ├───────────┼───────────────────────────┤
9003 │german │ org.apache.lucene.analy‐ │
9004 │ │ sis.de.GermanAnalyzer │
9005 ├───────────┼───────────────────────────┤
9006 │greek │ org.apache.lucene.analy‐ │
9007 │ │ sis.el.GreekAnalyzer │
9008 ├───────────┼───────────────────────────┤
9009 │galician │ org.apache.lucene.analy‐ │
9010 │ │ sis.gl.GalicianAnalyzer │
9011 ├───────────┼───────────────────────────┤
9012 │hindi │ org.apache.lucene.analy‐ │
9013 │ │ sis.hi.HindiAnalyzer │
9014 ├───────────┼───────────────────────────┤
9015 │hungarian │ org.apache.lucene.analy‐ │
9016 │ │ sis.hu.HungarianAnalyzer │
9017 ├───────────┼───────────────────────────┤
9018 │indonesian │ org.apache.lucene.analy‐ │
9019 │ │ sis.id.IndonesianAnalyzer │
9020 ├───────────┼───────────────────────────┤
9021 │irish │ org.apache.lucene.analy‐ │
9022 │ │ sis.ga.IrishAnalyzer │
9023 ├───────────┼───────────────────────────┤
9024 │italian │ org.apache.lucene.analy‐ │
9025 │ │ sis.it.ItalianAnalyzer │
9026 ├───────────┼───────────────────────────┤
9027 │japanese │ org.apache.lucene.analy‐ │
9028 │ │ sis.ja.JapaneseAnalyzer │
9029 ├───────────┼───────────────────────────┤
9030 │japanese │ org.apache.lucene.analy‐ │
9031 │ │ sis.ja.JapaneseTokenizer │
9032 └───────────┴───────────────────────────┘
9033 │latvian │ org.apache.lucene.analy‐ │
9036 │ │ sis.lv.LatvianAnalyzer │
9037 ├───────────┼───────────────────────────┤
9038 │norwegian │ org.apache.lucene.analy‐ │
9039 │ │ sis.no.NorwegianAnalyzer │
9040 ├───────────┼───────────────────────────┤
9041 │persian │ org.apache.lucene.analy‐ │
9042 │ │ sis.fa.PersianAnalyzer │
9043 ├───────────┼───────────────────────────┤
9044 │polish │ org.apache.lucene.analy‐ │
9045 │ │ sis.pl.PolishAnalyzer │
9046 ├───────────┼───────────────────────────┤
9047 │portuguese │ org.apache.lucene.analy‐ │
9048 │ │ sis.pt.PortugueseAnalyzer │
9049 ├───────────┼───────────────────────────┤
9050 │romanian │ org.apache.lucene.analy‐ │
9051 │ │ sis.ro.RomanianAnalyzer │
9052 ├───────────┼───────────────────────────┤
9053 │russian │ org.apache.lucene.analy‐ │
9054 │ │ sis.ru.RussianAnalyzer │
9055 ├───────────┼───────────────────────────┤
9056 │spanish │ org.apache.lucene.analy‐ │
9057 │ │ sis.es.SpanishAnalyzer │
9058 ├───────────┼───────────────────────────┤
9059 │swedish │ org.apache.lucene.analy‐ │
9060 │ │ sis.sv.SwedishAnalyzer │
9061 ├───────────┼───────────────────────────┤
9062 │thai │ org.apache.lucene.analy‐ │
9063 │ │ sis.th.ThaiAnalyzer │
9064 ├───────────┼───────────────────────────┤
9065 │turkish │ org.apache.lucene.analy‐ │
9066 │ │ sis.tr.TurkishAnalyzer │
9067 └───────────┴───────────────────────────┘
9068 NOTE:
9070 The japanese analyzer, org.apache.lucene.analysis.ja.JapaneseTok‐
9071 enizer, includes DEFAULT_MODE and defaultStopTags.
9072 NOTE:
9074 Language-specific analyzers are optimized for the specified lan‐
9075 guage. You cannot combine a generic analyzer with a language-spe‐
9076 cific analyzer. Instead, you might use a per field analyzer to
9077 select different analyzers for different fields within the docu‐
9078 ments.
9079 Per-field analyzers
9081 The perfield analyzer configures multiple analyzers for different
9082 fields.
9083 Example of defining different analyzers for different fields:
9085 {
9087 "_id": "_design/analyzer_example",
9088 "indexes": {
9089 "INDEX_NAME": {
9090 "analyzer": {
9091 "name": "perfield",
9092 "default": "english",
9093 "fields": {
9094 "spanish": "spanish",
9095 "german": "german"
9096 }
9097 },
9098 "index": "function (doc) { ... }"
9099 }
9100 }
9101 }
9102 Stop words
9104 Stop words are words that do not get indexed. You define them within a
9105 design document by turning the analyzer string into an object.
9106 NOTE:
9108 The keyword, simple, and whitespace analyzers do not support stop
9109 words.
9110 The default stop words for the standard analyzer are included below:
9112 "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if",
9114 "in", "into", "is", "it", "no", "not", "of", "on", "or", "such",
9115 "that", "the", "their", "then", "there", "these", "they", "this",
9116 "to", "was", "will", "with"
9117 Example of defining non-indexed (‘stop’) words:
9119 {
9121 "_id": "_design/stop_words_example",
9122 "indexes": {
9123 "INDEX_NAME": {
9124 "analyzer": {
9125 "name": "portuguese",
9126 "stopwords": [
9127 "foo",
9128 "bar",
9129 "baz"
9130 ]
9131 },
9132 "index": "function (doc) { ... }"
9133 }
9134 }
9135 }
9136 Testing analyzer tokenization
9138 You can test the results of analyzer tokenization by posting sample
9139 data to the _search_analyze endpoint.
9140 Example of using HTTP to test the keyword analyzer:
9142 POST /_search_analyze HTTP/1.1
9144 Content-Type: application/json
9145 {"analyzer":"keyword", "text":"ablanks@renovations.com"}
9146 Example of using the command line to test the keyword analyzer:
9148 curl 'https://$HOST:5984/_search_analyze' -H 'Content-Type: application/json'
9150 -d '{"analyzer":"keyword", "text":"ablanks@renovations.com"}'
9151 Result of testing the keyword analyzer:
9153 {
9155 "tokens": [
9156 "ablanks@renovations.com"
9157 ]
9158 }
9159 Example of using HTTP to test the standard analyzer:
9161 POST /_search_analyze HTTP/1.1
9163 Content-Type: application/json
9164 {"analyzer":"standard", "text":"ablanks@renovations.com"}
9165 Example of using the command line to test the standard analyzer:
9167 curl 'https://$HOST:5984/_search_analyze' -H 'Content-Type: application/json'
9169 -d '{"analyzer":"standard", "text":"ablanks@renovations.com"}'
9170 Result of testing the standard analyzer:
9172 {
9174 "tokens": [
9175 "ablanks",
9176 "renovations.com"
9177 ]
9178 }
9179 Queries
9181 After you create a search index, you can query it.
9182 · Issue a partition query using: GET /$DATABASE/_partition/$PARTI‐
9184 TION_KEY/_design/$DDOC/_search/$INDEX_NAME
9185 · Issue a global query using: GET /$DATA‐
9187 BASE/_design/$DDOC/_search/$INDEX_NAME
9188 Specify your search by using the query parameter.
9190 Example of using HTTP to query a partitioned index:
9192 GET /$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/_search/$INDEX_NAME?include_docs=true&query="*:*"&limit=1 HTTP/1.1
9194 Content-Type: application/json
9195 Example of using HTTP to query a global index:
9197 GET /$DATABASE/_design/$DDOC/_search/$INDEX_NAME?include_docs=true&query="*:*"&limit=1 HTTP/1.1
9199 Content-Type: application/json
9200 Example of using the command line to query a partitioned index:
9202 curl https://$HOST:5984/$DATABASE/_partition/$PARTITION_KEY/_design/$DDOC/
9204 _search/$INDEX_NAME?include_docs=true\&query="*:*"\&limit=1 \
9205 Example of using the command line to query a global index:
9207 curl https://$HOST:5984/$DATABASE/_design/$DDOC/_search/$INDEX_NAME?
9209 include_docs=true\&query="*:*"\&limit=1 \
9210 Query Parameters
9212 A full list of query parameters can be found in the API Reference.
9213 You must enable faceting before you can use the following parameters:
9215 · counts
9217 · drilldown
9219 · ranges
9221 NOTE:
9223 Do not combine the bookmark and stale options. These options con‐
9224 strain the choice of shard replicas to use for the response. When
9225 used together, the options might cause problems when contact is
9226 attempted with replicas that are slow or not available.
9227 Relevance
9229 When more than one result might be returned, it is possible for them to
9230 be sorted. By default, the sorting order is determined by ‘relevance’.
9231 Relevance is measured according to Apache Lucene Scoring. As an exam‐
9233 ple, if you search a simple database for the word example, two docu‐
9234 ments might contain the word. If one document mentions the word example
9235 10 times, but the second document mentions it only twice, then the
9236 first document is considered to be more ‘relevant’.
9237 If you do not provide a sort parameter, relevance is used by default.
9239 The highest scoring matches are returned first.
9240 If you provide a sort parameter, then matches are returned in that
9242 order, ignoring relevance.
9243 If you want to use a sort parameter, and also include ordering by rele‐
9245 vance in your search results, use the special fields -<score> or
9246 <score> within the sort parameter.
9247 POSTing search queries
9249 Instead of using the GET HTTP method, you can also use POST. The main
9250 advantage of POST queries is that they can have a request body, so you
9251 can specify the request as a JSON object. Each parameter in the query
9252 string of a GET request corresponds to a field in the JSON object in
9253 the request body.
9254 Example of using HTTP to POST a search request:
9256 POST /db/_design/ddoc/_search/searchname HTTP/1.1
9258 Content-Type: application/json
9259 Example of using the command line to POST a search request:
9261 curl 'https://$HOST:5984/db/_design/ddoc/_search/searchname' -X POST -H 'Content-Type: application/json' -d @search.json
9263 Example JSON document that contains a search request:
9265 {
9267 "q": "index:my query",
9268 "sort": "foo",
9269 "limit": 3
9270 }
9271 Query syntax
9273 The CouchDB search query syntax is based on the Lucene syntax. Search
9274 queries take the form of name:value unless the name is omitted, in
9275 which case they use the default field, as demonstrated in the following
9276 examples:
9277 Example search query expressions:
9279 // Birds
9281 class:bird
9282 // Animals that begin with the letter "l"
9284 l*
9285 // Carnivorous birds
9287 class:bird AND diet:carnivore
9288 // Herbivores that start with letter "l"
9290 l* AND diet:herbivore
9291 // Medium-sized herbivores
9293 min_length:[1 TO 3] AND diet:herbivore
9294 // Herbivores that are 2m long or less
9296 diet:herbivore AND min_length:[-Infinity TO 2]
9297 // Mammals that are at least 1.5m long
9299 class:mammal AND min_length:[1.5 TO Infinity]
9300 // Find "Meles meles"
9302 latin_name:"Meles meles"
9303 // Mammals who are herbivore or carnivore
9305 diet:(herbivore OR omnivore) AND class:mammal
9306 // Return all results
9308 *:*
9309 Queries over multiple fields can be logically combined, and groups and
9311 fields can be further grouped. The available logical operators are
9312 case-sensitive and are AND, +, OR, NOT and -. Range queries can run
9313 over strings or numbers.
9314 If you want a fuzzy search, you can run a query with ~ to find terms
9316 like the search term. For instance, look~ finds the terms book and
9317 took.
9318 NOTE:
9320 If the lower and upper bounds of a range query are both strings that
9321 contain only numeric digits, the bounds are treated as numbers not
9322 as strings. For example, if you search by using the query
9323 mod_date:["20170101" TO "20171231"], the results include documents
9324 for which mod_date is between the numeric values 20170101 and
9325 20171231, not between the strings “20170101” and “20171231”.
9326 You can alter the importance of a search term by adding ^ and a posi‐
9328 tive number. This alteration makes matches containing the term more or
9329 less relevant, proportional to the power of the boost value. The
9330 default value is 1, which means no increase or decrease in the strength
9331 of the match. A decimal value of 0 - 1 reduces importance. making the
9332 match strength weaker. A value greater than one increases importance,
9333 making the match strength stronger.
9334 Wildcard searches are supported, for both single (?) and multiple (*)
9336 character searches. For example, dat? would match date and data,
9337 whereas dat* would match date, data, database, and dates. Wildcards
9338 must come after the search term.
9339 Use *:* to return all results.
9341 If the search query does not specify the "group_field" argument, the
9343 response contains a bookmark. If this bookmark is later provided as a
9344 URL parameter, the response skips the rows that were seen already, mak‐
9345 ing it quick and easy to get the next set of results.
9346 NOTE:
9348 The response never includes a bookmark if the "group_field" parame‐
9349 ter is included in the search query. See group_field parameter.
9350 NOTE:
9352 The group_field, group_limit, and group_sort options are only avail‐
9353 able when making global queries.
9354 The following characters require escaping if you want to search on
9356 them:
9357 + - && || ! ( ) { } [ ] ^ " ~ * ? : \ /
9359 To escape one of these characters, use a preceding backslash character
9361 (\).
9362 The response to a search query contains an order field for each of the
9364 results. The order field is an array where the first element is the
9365 field or fields that are specified in the sort parameter. See the sort
9366 parameter. If no sort parameter is included in the query, then the
9367 order field contains the Lucene relevance score. If you use the ‘sort
9368 by distance’ feature as described in geographical searches, then the
9369 first element is the distance from a point. The distance is measured by
9370 using either kilometers or miles.
9371 NOTE:
9373 The second element in the order array can be ignored. It is used
9374 for troubleshooting purposes only.
9375 Faceting
9377 CouchDB Search also supports faceted searching, enabling discovery of
9378 aggregate information about matches quickly and easily. You can match
9379 all documents by using the special ?q=*:* query syntax, and use the
9380 returned facets to refine your query. To indicate that a field must be
9381 indexed for faceted queries, set {"facet": true} in its options.
9382 Example of search query, specifying that faceted search is enabled:
9384 function(doc) {
9386 index("type", doc.type, {"facet": true});
9387 index("price", doc.price, {"facet": true});
9388 }
9389 To use facets, all the documents in the index must include all the
9391 fields that have faceting enabled. If your documents do not include all
9392 the fields, you receive a bad_request error with the following reason,
9393 “The field_name does not exist.” If each document does not contain all
9394 the fields for facets, create separate indexes for each field. If you
9395 do not create separate indexes for each field, you must include only
9396 documents that contain all the fields. Verify that the fields exist in
9397 each document by using a single if statement.
9398 Example if statement to verify that the required fields exist in each
9400 document:
9401 if (typeof doc.town == "string" && typeof doc.name == "string") {
9403 index("town", doc.town, {facet: true});
9404 index("name", doc.name, {facet: true});
9405 }
9406 Counts
9408 NOTE:
9409 The counts option is only available when making global queries.
9410 The counts facet syntax takes a list of fields, and returns the number
9412 of query results for each unique value of each named field.
9413 NOTE:
9415 The count operation works only if the indexed values are strings.
9416 The indexed values cannot be mixed types. For example, if 100
9417 strings are indexed, and one number, then the index cannot be used
9418 for count operations. You can check the type by using the typeof
9419 operator, and convert it by using the parseInt, parseFloat, or
9420 .toString() functions.
9421 Example of a query using the counts facet syntax:
9423 ?q=*:*&counts=["type"]
9425 Example response after using of the counts facet syntax:
9427 {
9429 "total_rows":100000,
9430 "bookmark":"g...",
9431 "rows":[...],
9432 "counts":{
9433 "type":{
9434 "sofa": 10,
9435 "chair": 100,
9436 "lamp": 97
9437 }
9438 }
9439 }
9440 Drilldown
9442 NOTE:
9443 The drilldown option is only available when making global queries.
9444 You can restrict results to documents with a dimension equal to the
9446 specified label. Restrict the results by adding drilldown=["dimen‐
9447 sion","label"] to a search query. You can include multiple drilldown
9448 parameters to restrict results along multiple dimensions.
9449 Using a drilldown parameter is similar to using key:value in the q
9451 parameter, but the drilldown parameter returns values that the analyzer
9452 might skip.
9453 For example, if the analyzer did not index a stop word like "a", using
9455 drilldown returns it when you specify drilldown=["key","a"].
9456 Ranges
9458 NOTE:
9459 The ranges option is only available when making global queries.
9460 The range facet syntax reuses the standard Lucene syntax for ranges to
9462 return counts of results that fit into each specified category. Inclu‐
9463 sive range queries are denoted by brackets ([, ]). Exclusive range
9464 queries are denoted by curly brackets ({, }).
9465 NOTE:
9467 The range operation works only if the indexed values are numbers.
9468 The indexed values cannot be mixed types. For example, if 100
9469 strings are indexed, and one number, then the index cannot be used
9470 for range operations. You can check the type by using the typeof
9471 operator, and convert it by using the parseInt, parseFloat, or
9472 .toString() functions.
9473 Example of a request that uses faceted search for matching ranges:
9475 ?q=*:*&ranges={"price":{"cheap":"[0 TO 100]","expensive":"{100 TO Infinity}"}}
9477 Example results after a ranges check on a faceted search:
9479 {
9481 "total_rows":100000,
9482 "bookmark":"g...",
9483 "rows":[...],
9484 "ranges": {
9485 "price": {
9486 "expensive": 278682,
9487 "cheap": 257023
9488 }
9489 }
9490 }
9491 Geographical searches
9493 In addition to searching by the content of textual fields, you can also
9494 sort your results by their distance from a geographic coordinate using
9495 Lucene’s built-in geospatial capabilities.
9496 To sort your results in this way, you must index two numeric fields,
9498 representing the longitude and latitude.
9499 NOTE:
9501 You can also sort your results by their distance from a geographic
9502 coordinate using Lucene’s built-in geospatial capabilities.
9503 You can then query by using the special <distance...> sort field, which
9505 takes five parameters:
9506 · Longitude field name: The name of your longitude field (mylon in the
9508 example).
9509 · Latitude field name: The name of your latitude field (mylat in the
9511 example).
9512 · Longitude of origin: The longitude of the place you want to sort by
9514 distance from.
9515 · Latitude of origin: The latitude of the place you want to sort by
9517 distance from.
9518 · Units: The units to use: km for kilometers or mi for miles. The dis‐
9520 tance is returned in the order field.
9521 You can combine sorting by distance with any other search query, such
9523 as range searches on the latitude and longitude, or queries that
9524 involve non-geographical information.
9525 That way, you can search in a bounding box, and narrow down the search
9527 with extra criteria.
9528 Example geographical data:
9530 {
9532 "name":"Aberdeen, Scotland",
9533 "lat":57.15,
9534 "lon":-2.15,
9535 "type":"city"
9536 }
9537 Example of a design document that contains a search index for the geo‐
9539 graphic data:
9540 function(doc) {
9542 if (doc.type && doc.type == 'city') {
9543 index('city', doc.name, {'store': true});
9544 index('lat', doc.lat, {'store': true});
9545 index('lon', doc.lon, {'store': true});
9546 }
9547 }
9548 An example of using HTTP for a query that sorts cities in the northern
9550 hemisphere by their distance to New York:
9551 GET /examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>" HTTP/1.1
9553 An example of using the command line for a query that sorts cities in
9555 the northern hemisphere by their distance to New York:
9556 curl 'https://$HOST:5984/examples/_design/cities-designdoc/_search/cities?q=lat:[0+TO+90]&sort="<distance,lon,lat,-74.0059,40.7127,km>"'
9558 Example (abbreviated) response, containing a list of northern hemi‐
9560 sphere cities sorted by distance to New York:
9561 {
9563 "total_rows": 205,
9564 "bookmark": "g1A...XIU",
9565 "rows": [
9566 {
9567 "id": "city180",
9568 "order": [
9569 8.530665755719783,
9570 18
9571 ],
9572 "fields": {
9573 "city": "New York, N.Y.",
9574 "lat": 40.78333333333333,
9575 "lon": -73.96666666666667
9576 }
9577 },
9578 {
9579 "id": "city177",
9580 "order": [
9581 13.756343205985946,
9582 17
9583 ],
9584 "fields": {
9585 "city": "Newark, N.J.",
9586 "lat": 40.733333333333334,
9587 "lon": -74.16666666666667
9588 }
9589 },
9590 {
9591 "id": "city178",
9592 "order": [
9593 113.53603438866077,
9594 26
9595 ],
9596 "fields": {
9597 "city": "New Haven, Conn.",
9598 "lat": 41.31666666666667,
9599 "lon": -72.91666666666667
9600 }
9601 }
9602 ]
9603 }
9604 Highlighting search terms
9606 Sometimes it is useful to get the context in which a search term was
9607 mentioned so that you can display more emphasized results to a user.
9608 To get more emphasized results, add the highlight_fields parameter to
9610 the search query. Specify the field names for which you would like
9611 excerpts, with the highlighted search term returned.
9612 By default, the search term is placed in <em> tags to highlight it, but
9614 the highlight can be overridden by using the highlights_pre_tag and
9615 highlights_post_tag parameters.
9616 The length of the fragments is 100 characters by default. A different
9618 length can be requested with the highlights_size parameter.
9619 The highlights_number parameter controls the number of fragments that
9621 are returned, and defaults to 1.
9622 In the response, a highlights field is added, with one subfield per
9624 field name.
9625 For each field, you receive an array of fragments with the search term
9627 highlighted.
9628 NOTE:
9630 For highlighting to work, store the field in the index by using the
9631 store: true option.
9632 Example of using HTTP to search with highlighting enabled:
9634 GET /movies/_design/searches/_search/movies?q=movie_name:Azazel&highlight_fields=["movie_name"]&highlight_pre_tag="**"&highlight_post_tag="**"&highlights_size=30&highlights_number=2 HTTP/1.1
9636 Authorization: ...
9637 Example of using the command line to search with highlighting enabled:
9639 curl "https://$HOST:5984/movies/_design/searches/_search/movies?q=movie_name:Azazel&highlight_fields=\[\"movie_name\"\]&highlight_pre_tag=\"**\"&highlight_post_tag=\"**\"&highlights_size=30&highlights_number=2
9641 Example of highlighted search results:
9643 {
9645 "highlights": {
9646 "movie_name": [
9647 " on the Azazel Orient Express",
9648 " Azazel manuals, you"
9649 ]
9650 }
9651 }
9652 Note: Previously, the functionality provided by CouchDB’s design docu‐
9654 ments, in combination with document attachments, was referred to as
9655 “CouchApps.” The general principle was that entire web applications
9656 could be hosted in CouchDB, without need for an additional application
9657 server.
9658 Use of CouchDB as a combined standalone database and application server
9660 is no longer recommended. There are significant limitations to a pure
9661 CouchDB web server application stack, including but not limited to:
9662 fully-fledged fine-grained security, robust templating and scaffolding,
9663 complete developer tooling, and most importantly, a thriving ecosystem
9664 of developers, modules and frameworks to choose from.
9665 The developers of CouchDB believe that web developers should pick “the
9667 right tool for the right job”. Use CouchDB as your database layer, in
9668 conjunction with any number of other server-side web application frame‐
9669 works, such as the entire Node.JS ecosystem, Python’s Django and Flask,
9670 PHP’s Drupal, Java’s Apache Struts, and more.
9671
9673 In this chapter, we present some of the best ways to use Apache
9674 CouchDB. These usage patterns reflect many years of real-world use. We
9675 hope that these will jump-start your next project, or improve the per‐
9676 formance of your current system.
9677 Document Design Considerations
9679 When designing your database, and your document structure, there are a
9680 number of best practices to take into consideration. Especially for
9681 people accustomed to relational databases, some of these techniques may
9682 be non-obvious.
9683 Don’t rely on CouchDB’s auto-UUID generation
9685 While CouchDB will generate a unique identifier for the _id field of
9686 any doc that you create, in most cases you are better off generating
9687 them yourself for a few reasons:
9688 · If for any reason you miss the 200 OK reply from CouchDB, and storing
9690 the document is attempted again, you would end up with the same docu‐
9691 ment content stored under multiple _ids. This could easily happen
9692 with intermediary proxies and cache systems that may not inform
9693 developers that the failed transaction is being retried.
9694 · _ids are the only unique enforced value within CouchDB so you might
9696 as well make use of this. CouchDB stores its documents in a B+ tree.
9697 Each additional or updated document is stored as a leaf node, and may
9698 require re-writing intermediary and parent nodes. You may be able to
9699 take advantage of sequencing your own ids more effectively than the
9700 automatically generated ids if you can arrange them to be sequential
9701 yourself.
9702 Alternatives to auto-incrementing sequences
9704 Because of replication, as well as the distributed nature of CouchDB,
9705 it is not practical to use auto-incrementing sequences with CouchDB.
9706 These are often used to ensure unique identifiers for each row in a
9707 database table. CouchDB generates unique ids on its own and you can
9708 specify your own as well, so you don’t really need a sequence here. If
9709 you use a sequence for something else, you will be better off finding
9710 another way to express it in CouchDB in another way.
9711 Pre-aggregating your data
9713 If your intent for CouchDB is as a collect-and-report model, not a
9714 real-time view, you may not need to store a single document for every
9715 event you’re recording. In this case, pre-aggregating your data may be
9716 a good idea. You probably don’t need 1000 documents per second if all
9717 you are trying to do is to track summary statistics about those docu‐
9718 ments. This reduces the computational pressure on CouchDB’s MapReduce
9719 engine(s), as well as reduces its storage requirements.
9720 In this case, using an in-memory store to summarize your statistical
9722 information, then writing out to CouchDB every 10 seconds / 1 minute /
9723 whatever level of granularity you need would greatly reduce the number
9724 of documents you’ll put in your database.
9725 Later, you can then further decimate your data by walking the entire
9727 database and generating documents to be stored in a new database with a
9728 lower level of granularity (say, 1 document a day). You can then delete
9729 the older, more fine-grained database when you’re done with it.
9730 Designing an application to work with replication
9732 Whilst CouchDB includes replication and a conflict-flagging mechanism,
9733 this is not the whole story for building an application which repli‐
9734 cates in a way which users expect.
9735 Here we consider a simple example of a bookmarks application. The idea
9737 is that a user can replicate their own bookmarks, work with them on
9738 another machine, and then synchronise their changes later.
9739 Let’s start with a very simple definition of bookmarks: an ordered,
9741 nestable mapping of name to URL. Internally the application might rep‐
9742 resent it like this:
9743 [
9745 {"name":"Weather", "url":"http://www.bbc.co.uk/weather"},
9746 {"name":"News", "url":"http://news.bbc.co.uk/"},
9747 {"name":"Tech", "bookmarks": [
9748 {"name":"Register", "url":"http://www.theregister.co.uk/"},
9749 {"name":"CouchDB", "url":"http://couchdb.apache.org/"}
9750 ]}
9751 ]
9752 It can then present the bookmarks menu and sub-menus by traversing this
9754 structure.
9755 Now consider this scenario: the user has a set of bookmarks on her PC,
9757 and then replicates it to her laptop. On the laptop, she changes the
9758 News link to point to CNN, renames “Register” to “The Register”, and
9759 adds a new link to slashdot just after it. On the desktop, her husband
9760 deletes the Weather link, and adds a new link to CNET in the Tech
9761 folder.
9762 So after these changes, the laptop has:
9764 [
9766 {"name":"Weather", "url":"http://www.bbc.co.uk/weather"},
9767 {"name":"News", "url":"http://www.cnn.com/"},
9768 {"name":"Tech", "bookmarks": [
9769 {"name":"The Register", "url":"http://www.theregister.co.uk/"},
9770 {"name":"Slashdot", "url":"http://www.slashdot.new/"},
9771 {"name":"CouchDB", "url":"http://couchdb.apache.org/"}
9772 ]}
9773 ]
9774 and the PC has:
9776 [
9778 {"name":"News", "url":"http://www.cnn.com/"},
9779 {"name":"Tech", "bookmarks": [
9780 {"name":"Register", "url":"http://www.theregister.co.uk/"},
9781 {"name":"CouchDB", "url":"http://couchdb.apache.org/"},
9782 {"name":"CNET", "url":"http://news.cnet.com/"}
9783 ]}
9784 ]
9785 Upon the next synchronisation, we want the expected merge to take
9787 place. That is: links which were changed, added or deleted on one side
9788 are also changed, added or deleted on the other side - with no human
9789 intervention required unless absolutely necessary.
9790 We will also assume that both sides are doing a CouchDB “compact” oper‐
9792 ation periodically, and are disconnected for more than this time before
9793 they resynchronise.
9794 All of the approaches below which allow automated merging of changes
9796 rely on having some sort of history, back to the point where the repli‐
9797 cas diverged.
9798 CouchDB does not provide a mechanism for this itself. It stores arbi‐
9800 trary numbers of old _ids for one document (trunk now has a mechanism
9801 for pruning the _id history), for the purposes of replication. However
9802 it will not keep the documents themselves through a compaction cycle,
9803 except where there are conflicting versions of a document.
9804 Do not rely on the CouchDB revision history mechanism to help you build
9806 an application-level version history. Its sole purpose is to ensure
9807 eventually consistent replication between databases. It is up to you to
9808 maintain history explicitly in whatever form makes sense for your
9809 application, and to prune it to avoid excessive storage utilisation,
9810 whilst not pruning past the point where live replicas last diverged.
9811 Approach 1: Single JSON doc
9813 The above structure is already valid JSON, and so could be represented
9814 in CouchDB just by wrapping it in an object and storing as a single
9815 document:
9816 {
9818 "bookmarks":
9819 // ... same as above
9820 }
9821 This makes life very easy for the application, as the ordering and
9823 nesting is all taken care of. The trouble here is that on replication,
9824 only two sets of bookmarks will be visible: example B and example C.
9825 One will be chosen as the main revision, and the other will be stored
9826 as a conflicting revision.
9827 At this point, the semantics are very unsatisfactory from the user’s
9829 point of view. The best that can be offered is a choice saying “Which
9830 of these two sets of bookmarks do you wish to keep: B or C?” However
9831 neither represents the desired outcome. There is also insufficient data
9832 to be able to correctly merge them, since the base revision A is lost.
9833 This is going to be highly unsatisfactory for the user, who will have
9835 to apply one set of changes again manually.
9836 Approach 2: Separate document per bookmark
9838 An alternative solution is to make each field (bookmark) a separate
9839 document in its own right. Adding or deleting a bookmark is then just a
9840 case of adding or deleting a document, which will never conflict
9841 (although if the same bookmark is added on both sides, then you will
9842 end up with two copies of it). Changing a bookmark will only conflict
9843 if both sides made changes to the same one, and then it is reasonable
9844 to ask the user to choose between them.
9845 Since there will now be lots of small documents, you may either wish to
9847 keep a completely separate database for bookmarks, or else add an
9848 attribute to distinguish bookmarks from other kinds of document in the
9849 database. In the latter case, a view can be made to return only book‐
9850 mark documents.
9851 Whilst replication is now fixed, care is needed with the “ordered” and
9853 “nestable” properties of bookmarks.
9854 For ordering, one suggestion is to give each item a floating-point
9856 index, and then when inserting an object between A and B, give it an
9857 index which is the average of A and B’s indices. Unfortunately, this
9858 will fail after a while when you run out of precision, and the user
9859 will be bemused to find that their most recent bookmarks no longer
9860 remember the exact position they were put in.
9861 A better way is to keep a string representation of index, which can
9863 grow as the tree is subdivided. This will not suffer the above problem,
9864 but it may result in this string becoming arbitrarily long after time.
9865 They could be renumbered, but the renumbering operation could introduce
9866 a lot of conflicts, especially if attempted by both sides indepen‐
9867 dently.
9868 For “nestable”, you can have a separate doc which represents a list of
9870 bookmarks, and each bookmark can have a “belongs to” field which iden‐
9871 tifies the list. It may be useful anyway to be able to have multiple
9872 top-level bookmark sets (Bob’s bookmarks, Jill’s bookmarks etc). Some
9873 care is needed when deleting a list or sub-list, to ensure that all
9874 associated bookmarks are also deleted, otherwise they will become
9875 orphaned.
9876 Building the entire bookmark set can be performed through the use of
9878 emitting a compound key that describes the path to the document, then
9879 using group levels to retrieve the position of the tree in the docu‐
9880 ment. The following code excerpt describes a tree of files, where the
9881 path to the file is stored in the document under the "path" key:
9882 // map function
9884 function(doc) {
9885 if (doc.type === "file") {
9886 if (doc.path.substr(-1) === "/") {
9887 var raw_path = doc.path.slice(0, -1);
9888 } else {
9889 var raw_path = doc.path;
9890 }
9891 emit (raw_path.split('/'), 1);
9892 }
9893 }
9894 // reduce
9896 _sum
9897 This will emit rows into the view of the form ["opt", "couchdb", "etc",
9899 "local.ini"] for a doc.path of /opt/couchdb/etc/local.ini. You can then
9900 query a list of files in the /opt/couchdb/etc directory by specifying a
9901 startkey of ["opt", "couchdb", "etc"] and an endkey of ["opt",
9902 "couchdb", "etc", {}].
9903 Approach 3: Immutable history / event sourcing
9905 Another approach to consider is Event Sourcing or Command Logging, as
9906 implemented in many NoSQL databases and as used in many operational
9907 transformation systems.
9908 In this model, instead of storing individual bookmarks, you store
9910 records of changes made - “Bookmark added”, “Bookmark changed”, “Book‐
9911 mark moved”, “Bookmark deleted”. These are stored in an append-only
9912 fashion. Since records are never modified or deleted, only added to,
9913 there are never any replication conflicts.
9914 These records can also be stored as an array in a single CouchDB docu‐
9916 ment. Replication can cause a conflict, but in this case it is easy to
9917 resolve by simply combining elements from the two arrays.
9918 In order to see the full set of bookmarks, you need to start with a
9920 baseline set (initially empty) and run all the change records since the
9921 baseline was created; and/or you need to maintain a most-recent version
9922 and update it with changes not yet seen.
9923 Care is needed after replication when merging together history from
9925 multiple sources. You may get different results depending on how you
9926 order them - consider taking all A’s changes before B’s, taking all B’s
9927 before A’s, or interleaving them (e.g. if each change has a timestamp).
9928 Also, over time the amount of storage used can grow arbitrarily large,
9930 even if the set of bookmarks itself is small. This can be controlled by
9931 moving the baseline version forwards and then keeping only the changes
9932 after that point. However, care is needed not to move the baseline
9933 version forward so far that there are active replicas out there which
9934 last synchronised before that time, as this may result in conflicts
9935 which cannot be resolved automatically.
9936 If there is any uncertainty, it is best to present the user with a
9938 prompt to assist with merging the content in the application itself.
9939 Approach 4: Keep historic versions explicitly
9941 If you are going to keep a command log history, then it may be simpler
9942 just to keep old revisions of the bookmarks list itself around. The
9943 intention is to subvert CouchDB’s automatic behaviour of purging old
9944 revisions, by keeping these revisions as separate documents.
9945 You can keep a pointer to the ‘most current’ revision, and each revi‐
9947 sion can point to its predecessor. On replication, merging can take
9948 place by diffing each of the previous versions (in effect synthesising
9949 the command logs) back to a common ancestor.
9950 This is the sort of behaviour which revision control systems such as
9952 Git implement as a matter of routine, although generally comparing text
9953 files line-by-line rather than comparing JSON objects field-by-field.
9954 Systems like Git will accumulate arbitrarily large amounts of history
9956 (although they will attempt to compress it by packing multiple revi‐
9957 sions so that only their diffs are stored). With Git you can use “his‐
9958 tory rewriting” to remove old history, but this may prohibit merging if
9959 history doesn’t go back far enough in time.
9960 Adding client-side security with a translucent database
9962 Many applications do not require a thick layer of security at the
9963 server. It is possible to use a modest amount of encryption and one-way
9964 functions to obscure the sensitive columns or key-value pairs, a tech‐
9965 nique often called a translucent database. (See a description.)
9966 The simplest solutions use a one-way function like SHA-256 at the
9968 client to scramble the name and password before storing the informa‐
9969 tion. This solution gives the client control of the data in the data‐
9970 base without requiring a thick layer on the database to test each
9971 transaction. Some advantages are:
9972 · Only the client or someone with the knowledge of the name and pass‐
9974 word can compute the value of SHA256 and recover the data.
9975 · Some columns are still left in the clear, an advantage for computing
9977 aggregated statistics.
9978 · Computation of SHA256 is left to the client side computer which usu‐
9980 ally has cycles to spare.
9981 · The system prevents server-side snooping by insiders and any attacker
9983 who might penetrate the OS or any of the tools running upon it.
9984 There are limitations:
9986 · There is no root password. If the person forgets their name and pass‐
9988 word, their access is gone forever. This limits its use to databases
9989 that can continue by issuing a new user name and password.
9990 There are many variations on this theme detailed in the book
9992 Translucent Databases, including:
9993 · Adding a backdoor with public-key cryptography.
9995 · Adding a second layer with steganography.
9997 · Dealing with typographical errors.
9999 · Mixing encryption with one-way functions.
10001 Document submission using HTML Forms
10003 It is possible to write to a CouchDB document directly from an HTML
10004 form by using a document update function. Here’s how:
10005 The HTML form
10007 First, write an HTML form. Here’s a simple “Contact Us” form excerpt:
10008 <form action="/dbname/_design/ddocname/_update/contactform" method="post">
10010 <div>
10011 <label for="name">Name:</label>
10012 <input type="text" id="name" name="name" />
10013 </div>
10014 <div>
10015 <label for="mail">Email:</label>
10016 <input type="text" id="mail" name="email" />
10017 </div>
10018 <div>
10019 <label for="msg">Message:</label>
10020 <textarea id="msg" name="message"></textarea>
10021 </div>
10022 </form>
10023 Customize the /dbname/_design/ddocname/_update/contactform portion of
10025 the form action URL to reflect the exact path to your database, design
10026 document and update function (see below).
10027 As CouchDB no longer recommends the use of CouchDB-hosted web applica‐
10029 tions , you may want to use a reverse proxy to expose CouchDB as a sub‐
10030 directory of your web application. If so, add that prefix to the
10031 action destination in the form.
10032 Another option is to alter CouchDB’s CORS settings and use a
10034 cross-domain POST. Be sure you understand all security implications
10035 before doing this!
10036 The update function
10038 Then, write an update function. This is the server-side JavaScript
10039 function that will receive the POST-ed data.
10040 The first argument to the function will be the document that is being
10042 processed (if it exists). Because we are using POST and not PUT, this
10043 should be empty in our scenario - but we should check to be sure. The
10044 POST-ed data will be passed as the second parameter to the function,
10045 along with any query parameters and the full request headers.
10046 Here’s a sample handler that extracts the form data, generates a docu‐
10048 ment _id based on the email address and timestamp, and saves the docu‐
10049 ment. It then returns a JSON success response back to the browser.
10050 function(doc, req) {
10052 if (doc) {
10054 return [doc, toJSON({"error": "request already filed"})]
10055 }
10056 if !(req.form && req.form.email) {
10058 return [null, toJSON({"error": "incomplete form"})]
10059 }
10060 var date = new Date()
10062 var newdoc = req.form
10063 newdoc._id = req.form.email + "_" + date.toISOString()
10064 return [newdoc, toJSON({"success":"ok"})]
10066 }
10067 Place the above function in your design document under the updates key.
10069 Note that this function does not attempt any sort of input validation
10071 or sanitization. That is best handled by a validate document update
10072 function instead. (A “VDU” will validate any document written to the
10073 database, not just those that use your update function.)
10074 If the first element passed to return is a document, the HTTP response
10076 headers will include X-Couch-Id, the _id value for the newly created
10077 document, and X-Couch-Update-NewRev, the _rev value for the newly cre‐
10078 ated document. This is handy if your client-side code wants to access
10079 or update the document in a future call.
10080 Example output
10082 Here’s the worked sample above, using curl to simulate the form POST.
10083 $ curl -X PUT localhost:5984/testdb/_design/myddoc -d '{ "updates": { "contactform": "function(doc, req) { ... }" } }'
10085 {"ok":true,"id":"_design/myddoc","rev":"1-2a2b0951fcaf7287817573b03bba02ed"}
10086 $ curl --data "name=Lin&email=lin@example.com&message=I Love CouchDB" http://localhost:5984/testdb/_design/myddoc/_update/contactform
10088 * Trying 127.0.0.1...
10089 * TCP_NODELAY set
10090 * Connected to localhost (127.0.0.1) port 5984 (#1)
10091 > POST /testdb/_design/myddoc/_update/contactform HTTP/1.1
10092 > Host: localhost:5984
10093 > User-Agent: curl/7.59.0
10094 > Accept: */*
10095 > Content-Length: 53
10096 > Content-Type: application/x-www-form-urlencoded
10097 >
10098 * upload completely sent off: 53 out of 53 bytes
10099 < HTTP/1.1 201 Created
10100 < Content-Length: 16
10101 < Content-Type: text/html; charset=utf-8
10102 < Date: Thu, 05 Apr 2018 19:56:42 GMT
10103 < Server: CouchDB/2.2.0-948a1311c (Erlang OTP/19)
10104 < X-Couch-Id: lin%40example.com_2018-04-05T19:51:22.278Z
10105 < X-Couch-Request-ID: 03a5f4fbe0
10106 < X-Couch-Update-NewRev: 1-34483732407fcc6cfc5b60ace48b9da9
10107 < X-CouchDB-Body-Time: 0
10108 <
10109 * Connection #1 to host localhost left intact
10110 {"success":"ok"}
10111 $ curl http://localhost:5984/testdb/lin\@example.com_2018-04-05T19:51:22.278Z
10113 {"_id":"lin@example.com_2018-04-05T19:51:22.278Z","_rev":"1-34483732407fcc6cfc5b60ace48b9da9","name":"Lin","email":"lin@example.com","message":"I Love CouchDB"}
10114 Using an ISO Formatted Date for Document IDs
10116 The ISO 8601 date standard describes a useful scheme for representing a
10117 date string in a Year-Month-DayTHour:Minute:Second.microsecond format.
10118 For time-bound documents in a CouchDB database this can be a very handy
10119 way to create a unique identifier, since JavaScript can directly use it
10120 to create a Date object. Using this sample map function:
10121 function(doc) {
10123 var dt = new Date(doc._id);
10124 emit([dt.getDate(), doc.widget], 1);
10125 }
10126 simply use group_level to zoom in on whatever time you wish to use.
10128 curl -X GET "http://localhost:5984/transactions/_design/widget_count/_view/toss?group_level=1"
10130 {"rows":[
10132 {"key":[20],"value":10},
10133 {"key":[21],"value":20}
10134 ]}
10135 curl -X GET "http://localhost:5984/transactions/_design/widget_count/_view/toss?group_level=2"
10137 {"rows":[
10139 {"key":[20,widget],"value":10},
10140 {"key":[21,widget],"value":10},
10141 {"key":[21,thing],"value":10}
10142 ]}
10143 Another method is using parseint() and datetime.substr() to cut out
10145 useful values for a return key:
10146 function (doc) {
10148 var datetime = doc._id;
10149 var year = parseInt(datetime.substr(0, 4));
10150 var month = parseInt(datetime.substr(5, 2), 10);
10151 var day = parseInt(datetime.substr(8, 2), 10);
10152 var hour = parseInt(datetime.substr(11, 2), 10);
10153 var minute = parseInt(datetime.substr(14, 2), 10);
10154 emit([doc.widget, year, month, day, hour, minute], 1);
10155 }
10156 JavaScript development tips
10158 Working with Apache CouchDB’s JavaScript environment is a lot different
10159 than working with traditional JavaScript development environments. Here
10160 are some tips and tricks that will ease the difficulty.
10161 · Remember that CouchDB’s JavaScript engine is old, only supporting the
10163 ECMA-262 5th edition (“ES5”) of the language. ES6/2015 and newer con‐
10164 structs cannot be used.
10165 Fortunately, there are many tools available for transpiling modern
10167 JavaScript into code compatible with older JS engines. The Babel
10168 Project website, for example, offers an in-browser text editor which
10169 transpiles JavaScript in real-time. Configuring CouchDB-compatibility
10170 is as easy as enabling the ENV PRESET option, and typing “firefox
10171 4.0” into the Browsers field.
10172 · The log() function will log output to the CouchDB log file or stream.
10174 You can log strings, objects, and arrays directly, without first con‐
10175 verting to JSON. Use this in conjunction with a local CouchDB
10176 instance for best results.
10177 · Be sure to guard all document accesses to avoid exceptions when
10179 fields or subfields are missing: if (doc && doc.myarray &&
10180 doc.myarray.length)...
10181 View recommendations
10183 Here are some tips and tricks for working with CouchDB’s
10184 (JavaScript-based) views.
10185 Deploying a view change in a live environment
10187 It is possible to change the definition of a view, build the index,
10188 then make those changes go live without causing downtime for your
10189 application. The trick to making this work is that CouchDB’s JavaScript
10190 view index files are based on the contents of the design document - not
10191 its name, _id or revision. This means that two design documents with
10192 identical view code will share the same on-disk view index files.
10193 Here is a worked example, assuming your /db/_design/ddoc needs tobe
10195 updated.
10196 1. Upload the old design doc to /db/_design/ddoc-old (or copy the docu‐
10198 ment) if you want an easy way to rollback in case of problems. The
10199 ddoc-old document will reference the same view indexes already built
10200 for _design/ddoc.
10201 2. Upload the updated design doc to /db/_design/ddoc-new.
10203 3. Query a view in the new design document to trigger secondary index
10205 generation. You can track the indexing progress via the
10206 /_active_tasks endpoint, or through the fauxton web interface.
10207 4. When the index is done being built, re-upload the updated design
10209 document to /db/_design/ddoc (or copy the document). The ddoc docu‐
10210 ment will now reference the same view indexes already built for
10211 _design/ddoc-new.
10212 5. Delete /db/_design/ddoc-new and/or /db/_design/ddoc-old at your dis‐
10214 cretion. Don’t forget to trigger compact/views/cleanup to reclaim
10215 disk space after deleting ddoc-old.
10216 The COPY HTTP verb can be used to copy the design document with a sin‐
10218 gle command:
10219 curl -X COPY <URL of source design document> -H "Destination: <ID of destination design document>"
10221 Reverse Proxies
10223 Reverse proxying with HAProxy
10224 CouchDB recommends the use of HAProxy as a load balancer and reverse
10225 proxy. The team’s experience with using it in production has shown it
10226 to be superior for configuration and montioring capabilities, as well
10227 as overall performance.
10228 CouchDB’s sample haproxy configuration is present in the code reposi‐
10230 tory and release tarball as rel/haproxy.cfg. It is included below. This
10231 example is for a 3 node CouchDB cluster:
10232 global
10234 maxconn 512
10235 spread-checks 5
10236 defaults
10238 mode http
10239 log global
10240 monitor-uri /_haproxy_health_check
10241 option log-health-checks
10242 option httplog
10243 balance roundrobin
10244 option forwardfor
10245 option redispatch
10246 retries 4
10247 option http-server-close
10248 timeout client 150000
10249 timeout server 3600000
10250 timeout connect 500
10251 stats enable
10253 stats uri /_haproxy_stats
10254 # stats auth admin:admin # Uncomment for basic auth
10255 frontend http-in
10257 # This requires HAProxy 1.5.x
10258 # bind *:$HAPROXY_PORT
10259 bind *:5984
10260 default_backend couchdbs
10261 backend couchdbs
10263 option httpchk GET /_up
10264 http-check disable-on-404
10265 server couchdb1 x.x.x.x:5984 check inter 5s
10266 server couchdb2 x.x.x.x:5984 check inter 5s
10267 server couchdb2 x.x.x.x:5984 check inter 5s
10268 Reverse proxying with nginx
10270 Basic Configuration
10271 Here’s a basic excerpt from an nginx config file in <nginx config
10272 directory>/sites-available/default. This will proxy all requests from
10273 http://domain.com/... to http://localhost:5984/...
10274 location / {
10276 proxy_pass http://localhost:5984;
10277 proxy_redirect off;
10278 proxy_buffering off;
10279 proxy_set_header Host $host;
10280 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10281 }
10282 Proxy buffering must be disabled, or continuous replication will not
10284 function correctly behind nginx.
10285 Reverse proxying CouchDB in a subdirectory with nginx
10287 It can be useful to provide CouchDB as a subdirectory of your overall
10288 domain, especially to avoid CORS concerns. Here’s an excerpt of a basic
10289 nginx configuration that proxies the URL http://domain.com/couchdb to
10290 http://localhost:5984 so that requests appended to the subdirectory,
10291 such as http://domain.com/couchdb/db1/doc1 are proxied to http://local‐
10292 host:5984/db1/doc1.
10293 location /couchdb {
10295 rewrite /couchdb/(.*) /$1 break;
10296 proxy_pass http://localhost:5984;
10297 proxy_redirect off;
10298 proxy_buffering off;
10299 proxy_set_header Host $host;
10300 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10301 }
10302 Session based replication is default functionality since CouchDB 2.3.0.
10304 To enable session based replication with reverse proxied CouchDB in a
10305 subdirectory.
10306 location /_session {
10308 proxy_pass http://localhost:5984/_session;
10309 proxy_redirect off;
10310 proxy_buffering off;
10311 proxy_set_header Host $host;
10312 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10313 }
10314 Authentication with nginx as a reverse proxy
10316 Here’s a sample config setting with basic authentication enabled, plac‐
10317 ing CouchDB in the /couchdb subdirectory:
10318 location /couchdb {
10320 auth_basic "Restricted";
10321 auth_basic_user_file htpasswd;
10322 rewrite /couchdb/(.*) /$1 break;
10323 proxy_pass http://localhost:5984;
10324 proxy_redirect off;
10325 proxy_buffering off;
10326 proxy_set_header Host $host;
10327 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10328 proxy_set_header Authorization "";
10329 }
10330 This setup leans entirely on nginx performing authorization, and for‐
10332 warding requests to CouchDB with no authentication (with CouchDB in
10333 Admin Party mode), which isn’t sufficient in CouchDB 3.0 anymore as
10334 Admin Party has been removed. You’d need to at the very least
10335 hard-code user credentials into this version with headers.
10336 For a better solution, see api/auth/proxy.
10338 SSL with nginx
10340 In order to enable SSL, just enable the nginx SSL module, and add
10341 another proxy header:
10342 ssl on;
10344 ssl_certificate PATH_TO_YOUR_PUBLIC_KEY.pem;
10345 ssl_certificate_key PATH_TO_YOUR_PRIVATE_KEY.key;
10346 ssl_protocols SSLv3;
10347 ssl_session_cache shared:SSL:1m;
10348 location / {
10350 proxy_pass http://localhost:5984;
10351 proxy_redirect off;
10352 proxy_set_header Host $host;
10353 proxy_buffering off;
10354 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10355 proxy_set_header X-Forwarded-Ssl on;
10356 }
10357 The X-Forwarded-Ssl header tells CouchDB that it should use the https
10359 scheme instead of the http scheme. Otherwise, all CouchDB-generated
10360 redirects will fail.
10361 Reverse Proxying with Caddy
10363 Basic configuration
10364 Here’s a basic excerpt from a Caddyfile in /<path>/<to>/<site>/Caddy‐
10365 file. This will proxy all requests from http(s)://domain.com/... to
10366 http://localhost:5984/...
10367 domain.com {
10369 import /path/to/other/config.caddy # logging, error handling etc.
10371 proxy / localhost:5984 {
10373 transparent
10374 }
10375 }
10377 NOTE:
10379 The transparent preset in the proxy directive is shorthand for:
10380 header_upstream Host {host}
10382 header_upstream X-Real-IP {remote}
10383 header_upstream X-Forwarded-For {remote}
10384 header_upstream X-Forwarded-Proto {scheme}
10385 Note that, because Caddy is https-by-default, you must explicitly
10387 include the http:// protocol in the site address if you do NOT want
10388 Caddy to automatically acquire and install an SSL certificate and begin
10389 accepting https connections on port 443.
10390 Reverse proxying CouchDB in a subdirectory with Caddy
10392 It can be useful to provide CouchDB as a subdirectory of your overall
10393 domain, especially to avoid CORS concerns. Here’s an excerpt of a basic
10394 Caddy configuration that proxies the URL http(s)://domain.com/couchdb
10395 to http://localhost:5984 so that requests appended to the subdirectory,
10396 such as http(s)://domain.com/couchdb/db1/doc1 are proxied to
10397 http://localhost:5984/db1/doc1.
10398 domain.com {
10400 import /path/to/other/config.caddy # logging, error handling etc.
10402 proxy /couchdb localhost:5984 {
10404 transparent
10405 without /couchdb
10406 }
10407 }
10409 Reverse proxying + load balancing for CouchDB clusters
10411 Here’s a basic excerpt from a Caddyfile in /<path>/<to>/<site>/Caddy‐
10412 file. This will proxy and evenly distribute all requests from
10413 http(s)://domain.com/... among 3 CouchDB cluster nodes at local‐
10414 host:15984, localhost:25984 and localhost:35984.
10415 Caddy will check the status, i.e. health, of each node every 5 seconds;
10417 if a node goes down, Caddy will avoid proxying requests to that node
10418 until it comes back online.
10419 domain.com {
10421 import /path/to/other/config.caddy # logging, error handling etc.
10423 proxy / http://localhost:15984 http://localhost:25984 http://localhost:35984 {
10425 policy round_robin
10426 health_check /_up
10427 health_check_duration 5s
10428 try_interval 500ms
10429 keepalive 0
10430 transparent
10431 }
10432 }
10434 Authentication with Caddy as a reverse proxy
10436 Here’s a sample config setting with basic authentication enabled, plac‐
10437 ing CouchDB in the /couchdb subdirectory:
10438 domain.com {
10440 import /path/to/other/config.caddy # logging, error handling etc.
10442 basicauth /couchdb couch_username couchdb_password
10444 proxy /couchdb localhost:5984 {
10446 transparent
10447 header_upstream -Authorization
10448 without /couchdb
10449 }
10450 }
10452 For security reasons, using a plaintext password in the Caddyfile is
10454 not advisable. One solution is to define Caddy-process environment
10455 variables e.g. COUCH_PW=couchdb_password and using placeholders in the
10456 Caddyfile instead, e.g. {$COUCH_PW}.
10457 This setup leans entirely on nginx performing authorization, and for‐
10459 warding requests to CouchDB with no authentication (with CouchDB in
10460 Admin Party mode), which isn’t sufficient in CouchDB 3.0 anymore as
10461 Admin Party has been removed. You’d need to at the very least
10462 hard-code user credentials into this version with headers.
10463 For a better solution, see api/auth/proxy.
10465 SSL/TLS with Caddy
10467 Caddy is https-by-default, and will automatically acquire, install,
10468 activate and, when necessary, renew a trusted SSL certificate for you -
10469 all in the background. Certificates are issued by the LetsEncrypt cer‐
10470 tificate authority.
10471 domain.com {
10473 import /path/to/other/config.caddy # logging, error handling etc.
10475 proxy / localhost:5984 {
10477 transparent
10478 header_upstream x-forwarded-ssl on
10479 }
10480 }
10482 The x-forwarded-ssl header tells CouchDB that it should use the https
10484 scheme instead of the http scheme. Otherwise, all CouchDB-generated
10485 redirects will fail.
10486 Reverse Proxying with Apache HTTP Server
10488 WARNING:
10489 As of this writing, there is no way to fully disable the buffering
10490 between Apache HTTPD Server and CouchDB. This may present problems
10491 with continuous replication. The Apache CouchDB team strongly recom‐
10492 mend the use of an alternative reverse proxy such as haproxy or
10493 nginx, as described earlier in this section.
10494 Basic Configuration
10496 Here’s a basic excerpt for using a VirtualHost block config to use
10497 Apache as a reverse proxy for CouchDB. You need at least to configure
10498 Apache with the --enable-proxy --enable-proxy-http options and use a
10499 version equal to or higher than Apache 2.2.7 in order to use the
10500 nocanon option in the ProxyPass directive. The ProxyPass directive adds
10501 the X-Forwarded-For header needed by CouchDB, and the ProxyPreserveHost
10502 directive ensures the original client Host header is preserved.
10503 <VirtualHost *:80>
10505 ServerAdmin webmaster@dummy-host.example.com
10506 DocumentRoot "/opt/websites/web/www/dummy"
10507 ServerName couchdb.localhost
10508 AllowEncodedSlashes On
10509 ProxyRequests Off
10510 KeepAlive Off
10511 <Proxy *>
10512 Order deny,allow
10513 Deny from all
10514 Allow from 127.0.0.1
10515 </Proxy>
10516 ProxyPass / http://localhost:5984 nocanon
10517 ProxyPassReverse / http://localhost:5984
10518 ProxyPreserveHost On
10519 ErrorLog "logs/couchdb.localhost-error_log"
10520 CustomLog "logs/couchdb.localhost-access_log" common
10521 </VirtualHost>
10522
10524 Installation on Unix-like systems
10525 Installation using the Apache CouchDB convenience binary packages
10526 If you are running one of the following operating systems, the easiest
10527 way to install CouchDB is to use the convenience binary packages:
10528 · CentOS/RHEL 6
10530 · CentOS/RHEL 7
10532 · Debian 9 (stretch)
10534 · Debian 10 (buster)
10536 · Ubuntu 16.04 (xenial)
10538 · Ubuntu 18.04 (bionic)
10540 These RedHat-style rpm packages and Debian-style deb packages will
10542 install CouchDB at /opt/couchdb and ensure CouchDB is run at system
10543 startup by the appropriate init subsystem (SysV-style initd or sys‐
10544 temd).
10545 The Debian-style deb packages also pre-configure CouchDB as a stand‐
10547 alone or clustered node, prompt for the address to which it will bind,
10548 and a password for the admin user. Responses to these prompts may be
10549 pre-seeded using standard debconf tools. Further details are in the
10550 README.Debian file.
10551 Apache CouchDB also provides packages for the SpiderMonkey 1.8.5
10553 JavaScript dependency, as the upstream packages for this shared library
10554 are starting to disappear or become unreliable.
10555 Enabling the Apache CouchDB package repository
10557 Debian 9 (stretch): Run the following commands:
10558 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10560 $ echo "deb https://apache.bintray.com/couchdb-deb stretch main" \
10561 | sudo tee -a /etc/apt/sources.list.d/couchdb.list
10562 Debian 10 (buster): Run the following commands:
10564 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10566 $ echo "deb https://apache.bintray.com/couchdb-deb buster main" \
10567 | sudo tee -a /etc/apt/sources.list.d/couchdb.list
10568 Ubuntu 16.04 (Xenial): Run the following commands:
10570 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10572 $ echo "deb https://apache.bintray.com/couchdb-deb xenial main" \
10573 | sudo tee -a /etc/apt/sources.list.d/couchdb.list
10574 Ubuntu 18.04 (Bionic): Run the following commands:
10576 $ sudo apt-get install -y apt-transport-https gnupg ca-certificates
10578 $ echo "deb https://apache.bintray.com/couchdb-deb bionic main" \
10579 | sudo tee -a /etc/apt/sources.list.d/couchdb.list
10580 CentOS: Place the following text into /etc/yum.repos.d/bin‐
10582 tray-apache-couchdb-rpm.repo:
10583 [bintray--apache-couchdb-rpm]
10585 name=bintray--apache-couchdb-rpm
10586 baseurl=http://apache.bintray.com/couchdb-rpm/el$releasever/$basearch/
10587 gpgcheck=0
10588 repo_gpgcheck=0
10589 enabled=1
10590 RedHat 6: Place the following text into /etc/yum.repos.d/bin‐
10592 tray-apache-couchdb-rpm.repo:
10593 [bintray--apache-couchdb-rpm]
10595 name=bintray--apache-couchdb-rpm
10596 baseurl=http://apache.bintray.com/couchdb-rpm/el6/$basearch/
10597 gpgcheck=0
10598 repo_gpgcheck=0
10599 enabled=1
10600 RedHat 7: Place the following text into /etc/yum.repos.d/bin‐
10602 tray-apache-couchdb-rpm.repo:
10603 [bintray--apache-couchdb-rpm]
10605 name=bintray--apache-couchdb-rpm
10606 baseurl=http://apache.bintray.com/couchdb-rpm/el7/$basearch/
10607 gpgcheck=0
10608 repo_gpgcheck=0
10609 enabled=1
10610 Installing the Apache CouchDB packages
10612 Debian/Ubuntu: First, install the CouchDB repository key:
10613 $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys \
10615 8756C4F765C9AC3CB6B85D62379CE192D401AB61
10616 Then update the repository cache and install the package:
10618 $ sudo apt update
10620 $ sudo apt install -y couchdb
10621 Debian/Ubuntu installs from binaries can be pre-configured for single
10623 node or clustered installations. For clusters, multiple nodes will
10624 still need to be joined together and configured consistently across all
10625 machines; follow the Cluster Setup walkthrough to complete the process.
10626 RedHat/CentOS: Run the command:
10628 $ sudo yum -y install epel-release && sudo yum -y install couchdb
10630 Your installation is not complete. Be sure to complete the Setup steps
10632 for a single node or clustered installation.
10633 Relax! CouchDB is installed and running.
10635 Installation from source
10637 The remainder of this document describes the steps required to install
10638 CouchDB directly from source code.
10639 This guide, as well as the INSTALL.Unix document in the official tar‐
10641 ball release are the canonical sources of installation information.
10642 However, many systems have gotchas that you need to be aware of. In
10643 addition, dependencies frequently change as distributions update their
10644 archives.
10645 Dependencies
10647 You should have the following installed:
10648 · Erlang OTP (19.x, 20.x >= 21.3.8.5, 21.x >= 21.2.3, 22.x >= 22.0.5)
10650 · ICU
10652 · OpenSSL
10654 · Mozilla SpiderMonkey (1.8.5)
10656 · GNU Make
10658 · GNU Compiler Collection
10660 · libcurl
10662 · help2man
10664 · Python (>=2.7) for docs
10666 · Python Sphinx (>=1.1.3)
10668 It is recommended that you install Erlang OTP R16B03-1 or above where
10670 possible. You will only need libcurl if you plan to run the JavaScript
10671 test suite. And help2man is only need if you plan on installing the
10672 CouchDB man pages. Python and Sphinx are only required for building
10673 the online documentation. Documentation build can be disabled by
10674 adding the --disable-docs flag to the configure script.
10675 Debian-based Systems
10677 You can install the dependencies by running:
10678 sudo apt-get --no-install-recommends -y install \
10680 build-essential pkg-config erlang \
10681 libicu-dev libmozjs185-dev libcurl4-openssl-dev
10682 Be sure to update the version numbers to match your system’s available
10684 packages.
10685 RedHat-based (Fedora, Centos, RHEL) Systems
10687 You can install the dependencies by running:
10688 sudo yum install autoconf autoconf-archive automake \
10690 curl-devel erlang-asn1 erlang-erts erlang-eunit gcc-c++ \
10691 erlang-os_mon erlang-xmerl erlang-erl_interface help2man \
10692 js-devel-1.8.5 libicu-devel libtool perl-Test-Harness
10693 While CouchDB builds against the default js-devel-1.7.0 included in
10695 some distributions, it’s recommended to use a more recent
10696 js-devel-1.8.5.
10697 Warning: To build a release for CouchDB the erlang-reltool package is
10699 required, yet on CentOS/RHEL this package depends on erlang-wx which
10700 pulls in wxGTK and several X11 libraries. If CouchDB is being built on
10701 a console only server it might be a good idea to install this in a sep‐
10702 arate step to the rest of the dependencies, so that the package and all
10703 its dependencies can be removed using the yum history tool after the
10704 release is built. (reltool is needed only during release build but not
10705 for CouchDB functioning)
10706 The package can be installed by running:
10708 sudo yum install erlang-reltool
10710 Mac OS X
10712 Follow install/mac/homebrew reference for Mac App installation.
10713 If you are installing from source, you will need to install the Command
10715 Line Tools:
10716 xcode-select --install
10718 You can then install the other dependencies by running:
10720 brew install autoconf autoconf-archive automake libtool \
10722 erlang icu4c spidermonkey curl pkg-config
10723 You will need Homebrew installed to use the brew command.
10725 Some versions of Mac OS X ship a problematic OpenSSL library. If you’re
10727 experiencing troubles with CouchDB crashing intermittently with a seg‐
10728 mentation fault or a bus error, you will need to install your own ver‐
10729 sion of OpenSSL. See the wiki, mentioned above, for more information.
10730 SEE ALSO:
10732 · Homebrew
10734 FreeBSD
10736 FreeBSD requires the use of GNU Make. Where make is specified in this
10737 documentation, substitute gmake.
10738 You can install this by running:
10740 pkg install gmake
10742 Installing
10744 Once you have satisfied the dependencies you should run:
10745 ./configure
10747 If you wish to customize the installation, pass --help to this script.
10749 If everything was successful you should see the following message:
10751 You have configured Apache CouchDB, time to relax.
10753 Relax.
10755 To build CouchDB you should run:
10757 make release
10759 Try gmake if make is giving you any problems.
10761 If include paths or other compiler options must be specified, they can
10763 be passed to rebar, which compiles CouchDB, with the ERL_CFLAGS envi‐
10764 ronment variable. Likewise, options may be passed to the linker with
10765 the ERL_LDFLAGS environment variable:
10766 make release ERL_CFLAGS="-I/usr/local/include/js -I/usr/local/lib/erlang/usr/include"
10768 If everything was successful you should see the following message:
10770 ... done
10772 You can now copy the rel/couchdb directory anywhere on your system.
10773 Start CouchDB with ./bin/couchdb from within that directory.
10774 Relax.
10776 Note: a fully-fledged ./configure with the usual GNU Autotools options
10778 for package managers and a corresponding make install are in develop‐
10779 ment, but not part of the 2.0.0 release.
10780 User Registration and Security
10782 For OS X, in the steps below, substitute /Users/couchdb for
10783 /home/couchdb.
10784 You should create a special couchdb user for CouchDB.
10786 On many Unix-like systems you can run:
10788 adduser --system \
10790 --shell /bin/bash \
10791 --group --gecos \
10792 "CouchDB Administrator" couchdb
10793 On Mac OS X you can use the Workgroup Manager to create users up to
10795 version 10.9, and dscl or sysadminctl after version 10.9. Search
10796 Apple’s support site to find the documentation appropriate for your
10797 system. As of recent versions of OS X, this functionality is also
10798 included in Server.app, available through the App Store only as part of
10799 OS X Server.
10800 You must make sure that the user has a working POSIX shell and a
10802 writable home directory.
10803 You can test this by:
10805 · Trying to log in as the couchdb user
10807 · Running pwd and checking the present working directory
10809 As a recommendation, copy the rel/couchdb directory into /home/couchdb
10811 or /Users/couchdb.
10812 Ex: copy the built couchdb release to the new user’s home directory:
10814 cp -R /path/to/couchdb/rel/couchdb /home/couchdb
10816 Change the ownership of the CouchDB directories by running:
10818 chown -R couchdb:couchdb /home/couchdb
10820 Change the permission of the CouchDB directories by running:
10822 find /home/couchdb -type d -exec chmod 0770 {} \;
10824 Update the permissions for your ini files:
10826 chmod 0644 /home/couchdb/etc/*
10828 First Run
10830 You can start the CouchDB server by running:
10831 sudo -i -u couchdb /home/couchdb/bin/couchdb
10833 This uses the sudo command to run the couchdb command as the couchdb
10835 user.
10836 When CouchDB starts it should eventually display following messages:
10838 {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users" ...
10840 Don’t be afraid, we will fix this in a moment.
10842 To check that everything has worked, point your web browser to:
10844 http://127.0.0.1:5984/_utils/index.html
10846 From here you should verify your installation by pointing your web
10848 browser to:
10849 http://localhost:5984/_utils/index.html#verifyinstall
10851 Your installation is not complete. Be sure to complete the Setup steps
10853 for a single node or clustered installation.
10854 Running as a Daemon
10856 CouchDB no longer ships with any daemonization scripts.
10857 The CouchDB team recommends runit to run CouchDB persistently and reli‐
10859 ably. According to official site:
10860 runit is a cross-platform Unix init scheme with service supervision,
10861 a replacement for sysvinit, and other init schemes. It runs on
10862 GNU/Linux, *BSD, MacOSX, Solaris, and can easily be adapted to other
10863 Unix operating systems.
10864 Configuration of runit is straightforward; if you have questions, con‐
10866 tact the CouchDB user mailing list or IRC-channel #couchdb in FreeNode
10867 network.
10868 Let’s consider configuring runit on Ubuntu 16.04. The following steps
10870 should be considered only as an example. Details will vary by operating
10871 system and distribution. Check your system’s package management tools
10872 for specifics.
10873 Install runit:
10875 sudo apt-get install runit
10877 Create a directory where logs will be written:
10879 sudo mkdir /var/log/couchdb
10881 sudo chown couchdb:couchdb /var/log/couchdb
10882 Create directories that will contain runit configuration for CouchDB:
10884 sudo mkdir /etc/sv/couchdb
10886 sudo mkdir /etc/sv/couchdb/log
10887 Create /etc/sv/couchdb/log/run script:
10889 #!/bin/sh
10891 exec svlogd -tt /var/log/couchdb
10892 Basically it determines where and how exactly logs will be written.
10894 See man svlogd for more details.
10895 Create /etc/sv/couchdb/run:
10897 #!/bin/sh
10899 export HOME=/home/couchdb
10900 exec 2>&1
10901 exec chpst -u couchdb /home/couchdb/bin/couchdb
10902 This script determines how exactly CouchDB will be launched. Feel free
10904 to add any additional arguments and environment variables here if nec‐
10905 essary.
10906 Make scripts executable:
10908 sudo chmod u+x /etc/sv/couchdb/log/run
10910 sudo chmod u+x /etc/sv/couchdb/run
10911 Then run:
10913 sudo ln -s /etc/sv/couchdb/ /etc/service/couchdb
10915 In a few seconds runit will discover a new symlink and start CouchDB.
10917 You can control CouchDB service like this:
10918 sudo sv status couchdb
10920 sudo sv stop couchdb
10921 sudo sv start couchdb
10922 Naturally now CouchDB will start automatically shortly after system
10924 starts.
10925 You can also configure systemd, launchd or SysV-init daemons to launch
10927 CouchDB and keep it running using standard configuration files. Consult
10928 your system documentation for more information.
10929 Installation on Windows
10931 There are two ways to install CouchDB on Windows.
10932 Installation from binaries
10934 This is the simplest way to go.
10935 WARNING:
10937 Windows 10 requires the .NET Framwork v3.5 to be installed. You can
10938 install this via the Control Panel.
10939 1. Get the latest Windows binaries from the CouchDB web site. Old
10941 releases are available at archive.
10942 2. Follow the installation wizard steps. Be sure to install CouchDB to
10944 a path with no spaces, such as C:\CouchDB.
10945 3. Your installation is not complete. Be sure to complete the Setup
10947 steps for a single node or clustered installation.
10948 4. Open up Fauxton
10950 5. It’s time to Relax!
10952 NOTE:
10954 In some cases you might been asked to reboot Windows to complete
10955 installation process, because of using on different Microsoft Visual
10956 C++ runtimes by CouchDB.
10957 NOTE:
10959 Upgrading note
10960 It’s recommended to uninstall previous CouchDB version before
10962 upgrading, especially if the new one is built against different
10963 Erlang release. The reason is simple: there may be leftover
10964 libraries with alternative or incompatible versions from old Erlang
10965 release that may create conflicts, errors and weird crashes.
10966 In this case, make sure you backup of your local.ini config and
10968 CouchDB database/index files.
10969 Silent Install
10971 The Windows installer supports silent installs. Here are some sample
10972 commands, supporting the new features of the 3.0 installer.
10973 Install CouchDB without a service, but with an admin user:password of
10975 admin:hunter2:
10976 msiexec /i apache-couchdb-3.0.0.msi /quiet ADMINUSER=admin ADMINPASSWORD=hunter2 /norestart
10978 The same as above, but also install and launch CouchDB as a service:
10980 msiexec /i apache-couchdb-3.0.0.msi /quiet INSTALLSERVICE=1 ADMINUSER=admin ADMINPASSWORD=hunter2 /norestart
10982 Unattended uninstall of CouchDB:
10984 msiexec /x apache-couchdb-3.0.0.msi /quiet /norestart
10986 Unattended uninstall if the installer file is unavailable:
10988 msiexec /x {4CD776E0-FADF-4831-AF56-E80E39F34CFC} /quiet /norestart
10990 Add /l* log.txt to any of the above to generate a useful logfile for
10992 debugging.
10993 Installation from sources
10995 SEE ALSO:
10996 Glazier: Automate building of CouchDB from source on Windows
10997 Installation on macOS
10999 Installation using the Apache CouchDB native application
11000 The easiest way to run CouchDB on macOS is through the native macOS
11001 application. Just follow the below instructions:
11002 1. Download Apache CouchDB for macOS. Old releases are available at
11004 archive.
11005 2. Double click on the Zip file
11007 3. Drag and drop the Apache CouchDB.app into Applications folder
11009 That’s all, now CouchDB is installed on your Mac:
11011 1. Run Apache CouchDB application
11013 2. Open up Fauxton, the CouchDB admin interface
11015 3. Verify the install by clicking on Verify, then Verify Installation.
11017 4. Your installation is not complete. Be sure to complete the Setup
11019 steps for a single node or clustered installation.
11020 5. Time to Relax!
11022 Installation with Homebrew
11024 The Homebrew build of CouchDB 2.x is still in development. Check back
11025 often for updates.
11026 Installation from source
11028 Installation on macOS is possible from source. Download the source tar‐
11029 ball, extract it, and follow the instructions in the INSTALL.Unix.md
11030 file.
11031 Running as a Daemon
11033 CouchDB itself no longer ships with any daemonization scripts.
11034 The CouchDB team recommends runit to run CouchDB persistently and reli‐
11036 ably. Configuration of runit is straightforward; if you have questions,
11037 reach out to the CouchDB user mailing list.
11038 Naturally, you can configure launchd or other init daemons to launch
11040 CouchDB and keep it running using standard configuration files.
11041 Consult your system documentation for more information.
11043 Installation on FreeBSD
11045 Installation from ports
11046 cd /usr/ports/databases/couchdb
11047 make install clean
11048 This will install CouchDB from the ports collection.
11050 Start script
11052 The following options for /etc/rc.conf or /etc/rc.conf.local are sup‐
11053 ported by the start script (defaults shown):
11054 couchdb_enable="NO"
11056 couchdb_enablelogs="YES"
11057 couchdb_user="couchdb"
11058 After enabling the couchdb rc service use the following command to
11060 start CouchDB:
11061 /usr/local/etc/rc.d/couchdb start
11063 This script responds to the arguments start, stop, status, rcvar etc..
11065 The start script will also use settings from the following config
11067 files:
11068 · /usr/local/etc/couchdb/default.ini
11070 · /usr/local/etc/couchdb/local.ini
11072 Administrators should use default.ini as reference and only modify the
11074 local.ini file.
11075 Post install
11077 Your installation is not complete. Be sure to complete the Setup steps
11078 for a single node or clustered installation.
11079 In case the install script fails to install a non-interactive user
11081 “couchdb” to be used for the database, the user needs to be created
11082 manually:
11083 I used the pw command to add a user “couchdb” in group “couchdb”:
11085 pw user add couchdb
11087 pw user mod couchdb -c 'CouchDB, time to relax' -s /usr/sbin/nologin -d /var/lib/couchdb
11088 pw group add couchdb
11089 The user is added to /etc/passwd and should look similar to the follow‐
11091 ing:
11092 shell# grep couchdb /etc/passwd
11094 couchdb:*:1013:1013:Couchdb, time to relax:/var/lib/couchdb/:/usr/sbin/nologin
11095 To change any of these settings, please refrain from editing
11097 /etc/passwd and instead use pw user mod ... or vipw. Make sure that the
11098 user has no shell, but instead uses /usr/sbin/nologin. The ‘*’ in the
11099 second field means that this user can not login via password authoriza‐
11100 tion. For details use man 5 passwd.
11101 Installation via Docker
11103 Apache CouchDB provides ‘convenience binary’ Docker images through
11104 Docker Hub at apache/couchdb. The following tags are available:
11105 · latest, 2, 2.3, 2.3.1: CouchDB 2.3.1, single node
11107 · 1, 1.7, 1.7.2: CouchDB 1.7.2
11109 · 1-couchperuser, 1.7-couchperuser, 1.7.2-couchperuser: CouchDB 1.7.2
11111 with couchperuser plugin
11112 · 2.3.0: CouchDB 2.3.0, single node
11114 These images are built using Debian 9 (stretch), expose CouchDB on port
11116 5984 of the container, run everything as user couchdb, and support use
11117 of a Docker volume for data at /opt/couchdb/data.
11118 Note that you can also use the NODENAME environment variable to set the
11120 name of the CouchDB node inside the container.
11121 Your installation is not complete. Be sure to complete the Setup steps
11123 for a single node or clustered installation.
11124 Further details on the Docker configuration are available in our
11126 couchdb-docker git repository.
11127 Installation via Snap
11129 Apache CouchDB provides ‘convenience binary’ Snap builds through the
11130 Ubuntu snapcraft repository under the name couchdb. Only snaps built
11131 from official stable CouchDB releases (2.0, 2.1, etc.) are available
11132 through this channel.
11133 After installing snapd, the CouchDB snap can be installed via:
11135 $ sudo snap install couchdb
11137 CouchDB will be installed at /snap/couchdb. Data will be stored at
11139 /var/snap/couchdb/.
11140 Your installation is not complete. Be sure to complete the Setup steps
11142 for a single node or clustered installation.
11143 Further details on the snap build process are available in our
11145 couchdb-pkg git repository.
11146 Installation on Kubernetes
11148 Apache CouchDB provides a Helm chart to enable deployment to Kuber‐
11149 netes.
11150 To install the chart with the release name my-release:
11152 helm repo add couchdb https://apache.github.io/couchdb-helm
11154 helm repo update
11156 helm install --name my-release couchdb/couchdb
11158 Further details on the configuration options are available in the Helm
11160 chart readme.
11161 Search Plugin Installation
11163 New in version 3.0.
11164 CouchDB can build and query full-text search indexes using an external
11167 Java service that embeds Apache Lucene. Typically, this service is
11168 installed on the same host as CouchDB and communicates with it over the
11169 loopback network.
11170 The search plugin is runtime-compatible with Java JDKs 6, 7 and 8.
11172 Building a release from source requires JDK 6.
11173 Installation of Binary Packages
11175 Binary packages that bundle all the necessary dependencies of the
11176 search plugin are available on GitHub. The files in each release
11177 should be unpacked into a directory on the Java classpath. The service
11178 expects to find a couple of configuration files conventionally called
11179 clouseau.ini and log4j.properties with the following content:
11180 clouseau.ini:
11182 [clouseau]
11184 ; the name of the Erlang node created by the service, leave this unchanged
11186 name=clouseau@127.0.0.1
11187 ; set this to the same distributed Erlang cookie used by the CouchDB nodes
11189 cookie=monster
11190 ; the path where you would like to store the search index files
11192 dir=/path/to/index/storage
11193 ; the number of search indexes that can be open simultaneously
11195 max_indexes_open=500
11196 log4j.properties:
11198 log4j.rootLogger=debug, CONSOLE
11200 log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
11201 log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
11202 log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %c [%p] %m%n
11203 Once these files are in place the service can be started with an invo‐
11205 cation like the following:
11206 java -server \
11208 -Xmx2G \
11209 -Dsun.net.inetaddr.ttl=30 \
11210 -Dsun.net.inetaddr.negative.ttl=30 \
11211 -Dlog4j.configuration=file:/path/to/log4j.properties \
11212 -XX:OnOutOfMemoryError=\"kill -9 %p\" \
11213 -XX:+UseConcMarkSweepGC \
11214 -XX:+CMSParallelRemarkEnabled \
11215 com.cloudant.clouseau.Main \
11216 /path/to/clouseau.ini
11217 Chef
11219 The CouchDB cookbook can build the search plugin from source and
11220 install it on a server alongside CouchDB.
11221 Kubernetes
11223 Users running CouchDB on Kubernetes via the Helm chart can add the
11224 search service to each CouchDB Pod by setting enableSearch: true in the
11225 chart values.
11226 Additional Details
11228 The Search User Guide provides detailed information on creating and
11229 querying full-text indexes using this plugin.
11230 The source code for the plugin and additional configuration documenta‐
11232 tion is available on GitHub at
11233 https://github.com/cloudant-labs/clouseau.
11234 Upgrading from prior CouchDB releases
11236 Important Notes
11237 · Always back up your data/ and etc/ directories prior to upgrading
11238 CouchDB.
11239 · We recommend that you overwrite your etc/default.ini file with the
11241 version provided by the new release. New defaults sometimes contain
11242 mandatory changes to enable default functionality. Always places your
11243 customizations in etc/local.ini or any etc/local.d/*.ini file.
11244 Upgrading from CouchDB 2.x
11246 If you are coming from a prior release of CouchDB 2.x, upgrading is
11247 simple.
11248 Standalone (single) node upgrades
11250 If you are running a standalone (single) CouchDB node:
11251 1. Plan for downtime.
11253 2. Backup everything.
11255 3. Check for new recommended settings in the shipped etc/local.ini
11257 file, and merge any changes desired into your own local settings
11258 file(s).
11259 4. Stop CouchDB.
11261 5. Upgrade CouchDB in place.
11263 6. Start CouchDB.
11265 7. Relax! You’re done.
11267 Cluster upgrades
11269 CouchDB 2.x and 3.x are explicitly designed to allow “mixed clusters”
11270 during the upgrade process. This allows you to perform a rolling
11271 restart across a cluster, upgrading one node at a time, for a zero
11272 downtime upgrade. The process is also entirely scriptable within your
11273 configuration management tool of choice.
11274 We’re proud of this feature, and you should be, too!
11276 If you are running a CouchDB cluster:
11278 1. Backup everything.
11280 2. Check for new recommended settings in the shipped etc/local.ini
11282 file, and merge any changes desired into your own local settings
11283 file(s), staging these changes to occur as you upgrade the node.
11284 3. Stop CouchDB on a single node.
11286 4. Upgrade that CouchDB install in place.
11288 5. Start CouchDB.
11290 6. Double-check that the node has re-joined the cluster through the
11292 /_membership <api/server/membership> endpoint. If your load balancer
11293 has health check functionality driven by the /_up <api/server/up>
11294 endpoint, check whether it thinks the node is healthy as well.
11295 7. Repeat the last 4 steps on the remaining nodes in the cluster.
11297 8. Relax! You’re done.
11299 Upgrading from CouchDB 1.x
11301 To upgrade from CouchDB 1.x, first upgrade to a version of CouchDB 2.x.
11302 You will need to convert all databases to CouchDB 2.x format first; see
11303 the Upgrade Notes there for instructions. Then, upgrade to CouchDB 3.x.
11304 Troubleshooting an Installation
11306 First Install
11307 If your CouchDB doesn’t start after you’ve just installed, check the
11308 following things:
11309 · On UNIX-like systems, this is usually this is a permissions issue.
11311 Ensure that you’ve followed the install/unix/security chown/chmod
11312 commands. This problem is indicated by the presence of the keyword
11313 eacces somewhere in the error output from CouchDB itself.
11314 · Some Linux distributions split up Erlang into multiple packages. For
11316 your distribution, check that you really installed all the required
11317 Erlang modules. This varies from platform to platform, so you’ll just
11318 have to work it out for yourself. For example, on recent versions of
11319 Ubuntu/Debian, the erlang package includes all Erlang modules.
11320 · Confirm that Erlang itself starts up with crypto (SSL) support:
11322 ## what version of erlang are you running? Ensure it is supported
11324 erl -noshell -eval 'io:put_chars(erlang:system_info(otp_release)).' -s erlang halt
11325 ## are the erlang crypto (SSL) libraries working?
11326 erl -noshell -eval 'case application:load(crypto) of ok -> io:put_chars("yay_crypto\n") ; _ -> exit(no_crypto) end.' -s init stop
11327 · Next, identify where your Erlang CouchDB libraries are installed.
11329 This will typically be the lib/ subdirectory of the release that you
11330 have installed.
11331 · Use this to start up Erlang with the CouchDB libraries in its path:
11333 erl -env ERL_LIBS $ERL_LIBS:/path/to/couchdb/lib -couch_ini -s crypto
11335 · In that Erlang shell, let’s check that the key libraries are running.
11337 The %% lines are comments, so you can skip them:
11338 %% test SSL support. If this fails, ensure you have the OTP erlang-crypto library installed
11340 crypto:md5_init().
11341 %% test Snappy compression. If this fails, check your CouchDB configure script output or alternatively
11343 %% if your distro comes with erlang-snappy make sure you're using only the CouchDB supplied version
11344 snappy:compress("gogogogogogogogogogogogogogo").
11345 %% test the CouchDB JSON encoder. CouchDB uses different encoders in each release, this one matches
11347 %% what is used in 2.0.x.
11348 jiffy:decode(jiffy:encode(<<"[1,2,3,4,5]">>)).
11349 %% this is how you quit the erlang shell.
11351 q().
11352 · The output should resemble this, or an error will be thrown:
11354 Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]
11356 Eshell V6.2 (abort with ^G)
11358 1> crypto:md5_init().
11359 <<1,35,69,103,137,171,205,239,254,220,186,152,118,84,50,
11360 16,0,0,0,0,0,0,0,0,0,0,0,0,0,...>>
11361 2> snappy:compress("gogogogogogogogogogogogogogo").
11362 {ok,<<28,4,103,111,102,2,0>>}
11363 3> jiffy:decode(jiffy:encode(<<"[1,2,3,4,5]">>)).
11364 <<"[1,2,3,4,5]">>
11365 4> q().
11366 · At this point the only remaining dependency is your system’s Unicode
11368 support library, ICU, and the Spidermonkey Javascript VM from
11369 Mozilla. Make sure that your LD_LIBRARY_PATH or equivalent for
11370 non-Linux systems (DYLD_LIBRARY_PATH on macOS) makes these available
11371 to CouchDB. Linux example running as normal user:
11372 LD_LIBRARY_PATH=/usr/local/lib:/usr/local/spidermonkey/lib couchdb
11374 Linux example running as couchdb user:
11376 echo LD_LIBRARY_PATH=/usr/local/lib:/usr/local/spidermonkey/lib couchdb | sudo -u couchdb sh
11378 · If you receive an error message including the key word eaddrinuse,
11380 such as this:
11381 Failure to start Mochiweb: eaddrinuse
11383 edit your ``etc/default.ini`` or ``etc/local.ini`` file and change the
11385 ``[chttpd] port = 5984`` line to an available port.
11386 · If you receive an error including the string:
11388 … OS Process Error … {os_process_error,{exit_status,127}}
11390 then it is likely your SpiderMonkey JavaScript VM installation is not
11392 correct. Please recheck your build dependencies and try again.
11393 · If you receive an error including the string:
11395 … OS Process Error … {os_process_error,{exit_status,139}}
11397 this is caused by the fact that SELinux blocks access to certain areas
11399 of the file system. You must re-configure SELinux, or you can fully
11400 disable SELinux using the command:
11401 setenforce 0
11403 · If you are still not able to get CouchDB to start at this point, keep
11405 reading.
11406 Quick Build
11408 Having problems getting CouchDB to run for the first time? Follow this
11409 simple procedure and report back to the user mailing list or IRC with
11410 the output of each step. Please put the output of these steps into a
11411 paste service (such as https://paste.ee/) rather than including the
11412 output of your entire run in IRC or the mailing list directly.
11413 1. Note down the name and version of your operating system and your
11415 processor architecture.
11416 2. Note down the installed versions of CouchDB’s dependencies.
11418 3. Follow the checkout instructions to get a fresh copy of CouchDB’s
11420 trunk.
11421 4. Configure from the couchdb directory:
11423 ./configure
11425 5. Build the release:
11427 make release
11429 6. Run the couchdb command and log the output:
11431 cd rel/couchdb
11433 bin/couchdb
11434 7. Use your system’s kernel trace tool and log the output of the above
11436 command.
11437 a. For example, linux systems should use strace:
11439 strace bin/couchdb 2> strace.out
11441 8. Report back to the mailing list (or IRC) with the output of each
11443 step.
11444 Upgrading
11446 Are you upgrading from CouchDB 1.x? Install CouchDB into a fresh direc‐
11447 tory. CouchDB’s directory layout has changed and may be confused by
11448 libraries present from previous releases.
11449 Runtime Errors
11451 Lots of memory being used on startup
11452 Is your CouchDB using a lot of memory (several hundred MB) on startup?
11453 This one seems to especially affect Dreamhost installs. It’s really an
11454 issue with the Erlang VM pre-allocating data structures when ulimit is
11455 very large or unlimited. A detailed discussion can be found on the
11456 erlang-questions list, but the short answer is that you should decrease
11457 ulimit -n or define ERL_MAX_PORTS to something reasonable like 1024.
11458 erlang stack trace contains system_limit, open_port
11460 Erlang has a default limit of 1024 ports, where each FD, tcp connec‐
11461 tion, and linked-in driver uses one port. You seem to have exceeded
11462 this. You can change it at runtime using the ERL_MAX_PORTS env vari‐
11463 able.
11464 function raised exception (Cannot encode ‘undefined’ value as JSON)
11466 If you see this in the CouchDB error logs, the JavaScript code you are
11467 using for either a map or reduce function is referencing an object mem‐
11468 ber that is not defined in at least one document in your database. Con‐
11469 sider this document:
11470 {
11472 "_id":"XYZ123",
11473 "_rev":"1BB2BB",
11474 "field":"value"
11475 }
11476 and this map function:
11478 function(doc) {
11480 emit(doc.name, doc.address);
11481 }
11482 This will fail on the above document, as it does not contain a name or
11484 address member. Instead, use guarding to make sure the function only
11485 accesses members when they exist in a document:
11486 function(doc) {
11488 if(doc.name && doc.address) {
11489 emit(doc.name, doc.address);
11490 }
11491 }
11492 While the above guard will work in most cases, it’s worth bearing
11494 JavaScript’s understanding of ‘false’ values in mind. Testing against a
11495 property with a value of 0 (zero), '' (empty String), false or null
11496 will return false. If this is undesired, a guard of the form if
11497 (doc.foo !== undefined) should do the trick.
11498 This error can also be caused if a reduce function does not return a
11500 value. For example, this reduce function will cause an error:
11501 function(key, values) {
11503 sum(values);
11504 }
11505 The function needs to return a value:
11507 function(key, values) {
11509 return sum(values);
11510 }
11511 erlang stack trace contains bad_utf8_character_code
11513 CouchDB 1.1.1 and later contain stricter handling of UTF8 encoding. If
11514 you are replicating from older versions to newer versions, then this
11515 error may occur during replication.
11516 A number of work-arounds exist; the simplest is to do an in-place
11518 upgrade of the relevant CouchDB and then compact prior to replicating.
11519 Alternatively, if the number of documents impacted is small, use fil‐
11521 tered replication to exclude only those documents.
11522 FIPS mode
11524 Operating systems can be configured to disallow the use of OpenSSL MD5
11525 hash functions in order to prevent use of MD5 for cryptographic pur‐
11526 poses. CouchDB makes use of MD5 hashes for verifying the integrity of
11527 data (and not for cryptography) and will not run without the ability to
11528 use MD5 hashes.
11529 The message below indicates that the operating system is running in
11531 “FIPS mode,” which, among other restrictions, does not allow the use of
11532 OpenSSL’s MD5 functions:
11533 md5_dgst.c(82): OpenSSL internal error, assertion failed: Digest MD5 forbidden in FIPS mode!
11535 [os_mon] memory supervisor port (memsup): Erlang has closed
11536 [os_mon] cpu supervisor port (cpu_sup): Erlang has closed
11537 Aborted
11538 A workaround for this is provided with the --erlang-md5 compile flag.
11540 Use of the flag results in CouchDB substituting the OpenSSL MD5 func‐
11541 tion calls with equivalent calls to Erlang’s built-in library
11542 erlang:md5. NOTE: there may be a performance penalty associated with
11543 this workaround.
11544 Because CouchDB does not make use of MD5 hashes for cryptographic pur‐
11546 poses, this workaround does not defeat the purpose of “FIPS mode,” pro‐
11547 vided that the system owner is aware of and consents to its use.
11548 Debugging startup
11550 If you’ve compiled from scratch and are having problems getting CouchDB
11551 to even start up, you may want to see more detail. Start by enabling
11552 logging at the debug level:
11553 [log]
11555 level = debug
11556 You can then pass the -init_debug +W i +v +V -emu_args flags in the
11558 ERL_FLAGS environment variable to turn on additional debugging informa‐
11559 tion that CouchDB developers can use to help you.
11560 Then, reach out to the CouchDB development team using the links pro‐
11562 vided on the CouchDB home page for assistance.
11563 macOS Known Issues
11565 undefined error, exit_status 134
11566 Sometimes the Verify Installation fails with an undefined error. This
11567 could be due to a missing dependency with Mac. In the logs, you will
11568 find couchdb exit_status,134.
11569 Installing the missing nspr via brew install nspr resolves the issue.
11571 (see: https://github.com/apache/couchdb/issues/979)
11572
11574 CouchDB 2.x can be deployed in either a single-node or a clustered con‐
11575 figuration. This section covers the first-time setup steps required for
11576 each of these configurations.
11577 Single Node Setup
11579 Many users simply need a single-node CouchDB 2.x installation. Opera‐
11580 tionally, it is roughly equivalent to the CouchDB 1.x series. Note that
11581 a single-node setup obviously doesn’t take any advantage of the new
11582 scaling and fault-tolerance features in CouchDB 2.x.
11583 After installation and initial startup, visit Fauxton at
11585 http://127.0.0.1:5984/_utils#setup. You will be asked to set up CouchDB
11586 as a single-node instance or set up a cluster. When you click “Sin‐
11587 gle-Node-Setup”, you will get asked for an admin username and password.
11588 Choose them well and remember them.
11589 You can also bind CouchDB to a public address, so it is accessible
11591 within your LAN or the public, if you are doing this on a public VM.
11592 Or, you can keep the installation private by binding only to 127.0.0.1
11593 (localhost). Binding to 0.0.0.0 will bind to all addresses. The wizard
11594 then configures your admin username and password and creates the three
11595 system databases _users, _replicator and _global_changes for you.
11596 Another option is to set the configuration parameter [couchdb] sin‐
11598 gle_node=true in your local.ini file. When doing this, CouchDB will
11599 create the system database for you on restart.
11600 Alternatively, if you don’t want to use the Setup Wizard or set that
11602 value, and run 3.x as a single node with a server administrator already
11603 configured via config file, make sure to create the three system data‐
11604 bases manually on startup:
11605 curl -X PUT http://127.0.0.1:5984/_users
11607 curl -X PUT http://127.0.0.1:5984/_replicator
11609 curl -X PUT http://127.0.0.1:5984/_global_changes
11611 Note that the last of these is not necessary if you do not expect to be
11613 using the global changes feed. Feel free to delete this database if you
11614 have created it, it has grown in size, and you do not need the function
11615 (and do not wish to waste system resources on compacting it regularly.)
11616 Cluster Set Up
11618 This section describes everything you need to know to prepare, install,
11619 and set up your first CouchDB 2.x cluster.
11620 Ports and Firewalls
11622 CouchDB uses the following ports:
11623 ┌─────────────────┬──────────┬──────────────────┬──────────────────┐
11625 │Port Number │ Protocol │ Recommended │ Usage │
11626 │ │ │ binding │ │
11627 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11628 │5984 │ tcp │ As desired, by │ Standard clus‐ │
11629 │ │ │ default local‐ │ tered port for │
11630 │ │ │ host │ all HTTP API │
11631 │ │ │ │ requests │
11632 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11633 │4369 │ tcp │ All interfaces │ Erlang port map‐ │
11634 │ │ │ by default │ per daemon │
11635 │ │ │ │ (epmd) │
11636 ├─────────────────┼──────────┼──────────────────┼──────────────────┤
11637 │Random above │ tcp │ Automatic │ Communication │
11638 │1024 (see below) │ │ │ with other │
11639 │ │ │ │ CouchDB nodes in │
11640 │ │ │ │ the cluster │
11641 └─────────────────┴──────────┴──────────────────┴──────────────────┘
11642 CouchDB in clustered mode uses the port 5984, just as in a standalone
11644 configuration. Port 5986, previously used in CouchDB 2.x, has been
11645 removed in CouchDB 3.0. All endpoints previously accessible at that
11646 port are now available under the /_node/{node-name}/... hierarchy via
11647 the primary 5984 port.
11648 CouchDB uses Erlang-native clustering functionality to achieve a clus‐
11650 tered installation. Erlang uses TCP port 4369 (EPMD) to find other
11651 nodes, so all servers must be able to speak to each other on this port.
11652 In an Erlang cluster, all nodes are connected to all other nodes, in a
11653 mesh network configuration.
11654 WARNING:
11656 If you expose the port 4369 to the Internet or any other untrusted
11657 network, then the only thing protecting you is the Erlang cookie.
11658 Every Erlang application running on that machine (such as CouchDB) then
11660 uses automatically assigned ports for communciation with other nodes.
11661 Yes, this means random ports. This will obviously not work with a fire‐
11662 wall, but it is possible to force an Erlang application to use a spe‐
11663 cific port range.
11664 This documentation will use the range TCP 9100-9200, but this range is
11666 unnecessarily broad. If you only have a single Erlang application run‐
11667 ning on a machine, the range can be limited to a single port:
11668 9100-9100, since the ports epmd assign are for inbound connections
11669 only. Three CouchDB nodes running on a single machine, as in a develop‐
11670 ment cluster scenario, would need three ports in this range.
11671 Configure and Test the Communication with Erlang
11673 Make CouchDB use correct IP|FQDN and the open ports
11674 In file etc/vm.args change the line -name couchdb@127.0.0.1 to -name
11675 couchdb@<reachable-ip-address|fully-qualified-domain-name> which
11676 defines the name of the node. Each node must have an identifier that
11677 allows remote systems to talk to it. The node name is of the form
11678 <name>@<reachable-ip-address|fully-qualified-domain-name>.
11679 The name portion can be couchdb on all nodes, unless you are running
11681 more than 1 CouchDB node on the same server with the same IP address or
11682 domain name. In that case, we recommend names of couchdb1, couchdb2,
11683 etc.
11684 The second portion of the node name must be an identifier by which
11686 other nodes can access this node – either the node’s fully qualified
11687 domain name (FQDN) or the node’s IP address. The FQDN is preferred so
11688 that you can renumber the node’s IP address without disruption to the
11689 cluster. (This is common in cloud-hosted environments.)
11690 WARNING:
11692 Tricks with /etc/hosts and libresolv don’t work with Erlang. Either
11693 properly set up DNS and use fully-qualified domain names, or use IP
11694 addresses. DNS and FQDNs are preferred.
11695 Open etc/vm.args, on all nodes, and add -kernel inet_dist_listen_min
11697 9100 and -kernel inet_dist_listen_max 9200 like below:
11698 -name ...
11700 -setcookie ...
11701 ...
11702 -kernel inet_dist_listen_min 9100
11703 -kernel inet_dist_listen_max 9200
11704 Again, a small range is fine, down to a single port (set both to 9100)
11706 if you only ever run a single CouchDB node on each machine.
11707 Confirming connectivity between nodes
11709 For this test, you need 2 servers with working hostnames. Let us call
11710 them server1.test.com and server2.test.com. They reside at 192.168.0.1
11711 and 192.168.0.2, respectively.
11712 On server1.test.com:
11714 erl -name bus@192.168.0.1 -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200
11716 Then on server2.test.com:
11718 erl -name car@192.168.0.2 -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200
11720 An explanation to the commands:
11722 · erl the Erlang shell.
11724 · -name bus@192.168.0.1 the name of the Erlang node and its IP
11726 address or FQDN.
11727 · -setcookie 'brumbrum' the “password” used when nodes connect
11729 to each other.
11730 · -kernel inet_dist_listen_min 9100 the lowest port in the
11732 range.
11733 · -kernel inet_dist_listen_max 9200 the highest port in the
11735 range.
11736 This gives us 2 Erlang shells. shell1 on server1, shell2 on server2.
11738 Time to connect them. Enter the following, being sure to end the line
11739 with a period (.):
11740 In shell1:
11742 net_kernel:connect_node(car@192.168.0.2).
11744 This will connect to the node called car on the server called
11746 192.168.0.2.
11747 If that returns true, then you have an Erlang cluster, and the fire‐
11749 walls are open. This means that 2 CouchDB nodes on these two servers
11750 will be able to communicate with each other successfully. If you get
11751 false or nothing at all, then you have a problem with the firewall,
11752 DNS, or your settings. Try again.
11753 If you’re concerned about firewall issues, or having trouble connecting
11755 all nodes of your cluster later on, repeat the above test between all
11756 pairs of servers to confirm connectivity and system configuration is
11757 correct.
11758 Preparing CouchDB nodes to be joined into a cluster
11760 Before you can add nodes to form a cluster, you must have them listen‐
11761 ing on an IP address accessible from the other nodes in the cluster.
11762 You should also ensure that a few critical settings are identical
11763 across all nodes before joining them.
11764 The settings we recommend you set now, before joining the nodes into a
11766 cluster, are:
11767 1. etc/vm.args settings as described in the previous two sections
11769 2. At least one server administrator user (and password)
11771 3. Bind the node’s clustered interface (port 5984) to a reachable IP
11773 address
11774 4. A consistent UUID. The UUID is used in identifying the cluster when
11776 replicating. If this value is not consistent across all nodes in the
11777 cluster, replications may be forced to rewind the changes feed to
11778 zero, leading to excessive memory, CPU and network use.
11779 5. A consistent httpd secret. The secret is used in calculating and
11781 evaluating cookie and proxy authentication, and should be set con‐
11782 sistently to avoid unnecessary repeated session cookie requests.
11783 As of CouchDB 3.0, steps 4 and 5 above are automatically performed for
11785 you when using the setup API endpoints described below.
11786 If you use a configuration management tool, such as Chef, Ansible, Pup‐
11788 pet, etc., then you can place these settings in a .ini file and dis‐
11789 tribute them to all nodes ahead of time. Be sure to pre-encrypt the
11790 password (cutting and pasting from a test instance is easiest) if you
11791 use this route to avoid CouchDB rewriting the file.
11792 If you do not use configuration management, or are just experimenting
11794 with CouchDB for the first time, use these commands once per server to
11795 perform steps 2-4 above. Be sure to change the password to something
11796 secure, and again, use the same password on all nodes. You may have to
11797 run these commands locally on each node; if so, replace
11798 <server-IP|FQDN> below with 127.0.0.1.
11799 # First, get two UUIDs to use later on. Be sure to use the SAME UUIDs on all nodes.
11801 curl http://<server-IP|FQDN>:5984/_uuids?count=2
11802 # CouchDB will respond with something like:
11804 # {"uuids":["60c9e8234dfba3e2fdab04bf92001142","60c9e8234dfba3e2fdab04bf92001cc2"]}
11805 # Copy the provided UUIDs into your clipboard or a text editor for later use.
11806 # Use the first UUID as the cluster UUID.
11807 # Use the second UUID as the cluster shared http secret.
11808 # Create the admin user and password:
11810 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/admins/admin -d '"password"'
11811 # Now, bind the clustered interface to all IP addresses availble on this machine
11813 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/chttpd/bind_address -d '"0.0.0.0"'
11814 # If not using the setup wizard / API endpoint, the following 2 steps are required:
11816 # Set the UUID of the node to the first UUID you previously obtained:
11817 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/couchdb/uuid -d '"FIRST-UUID-GOES-HERE"'
11818 # Finally, set the shared http secret for cookie creation to the second UUID:
11820 curl -X PUT http://<server-IP|FQDN>:5984/_node/_local/_config/couch_httpd_auth/secret -d '"SECOND-UUID-GOES-HERE"'
11821 The Cluster Setup Wizard
11823 CouchDB 2.x comes with a convenient Cluster Setup Wizard as part of the
11824 Fauxton web administration interface. For first-time cluster setup, and
11825 for experimentation, this is your best option.
11826 It is strongly recommended that the minimum number of nodes in a clus‐
11828 ter is 3. For more explanation, see the Cluster Theory section of this
11829 documentation.
11830 After installation and initial start-up of all nodes in your cluster,
11832 ensuring all nodes are reachable, and the pre-configuration steps
11833 listed above, visit Fauxton at http://<server1>:5984/_utils#setup. You
11834 will be asked to set up CouchDB as a single-node instance or set up a
11835 cluster.
11836 When you click “Setup Cluster” you are asked for admin credentials
11838 again, and then to add nodes by IP address. To get more nodes, go
11839 through the same install procedure on other machines. Be sure to spec‐
11840 ify the total number of nodes you expect to add to the cluster before
11841 adding nodes.
11842 Now enter each node’s IP address or FQDN in the setup wizard, ensuring
11844 you also enter the previously set server admin username and password.
11845 Once you have added all nodes, click “Setup” and Fauxton will finish
11847 the cluster configuration for you.
11848 To check that all nodes have been joined correctly, visit
11850 http://<server-IP|FQDN>:5984/_membership on each node. The returned
11851 list should show all of the nodes in your cluster:
11852 {
11854 "all_nodes": [
11855 "couchdb@server1.test.com",
11856 "couchdb@server2.test.com",
11857 "couchdb@server3.test.com"
11858 ],
11859 "cluster_nodes": [
11860 "couchdb@server1.test.com",
11861 "couchdb@server2.test.com",
11862 "couchdb@server3.test.com"
11863 ]
11864 }
11865 The all_nodes section is the list of expected nodes; the cluster_nodes
11867 section is the list of actually connected nodes. Be sure the two lists
11868 match.
11869 Now your cluster is ready and available! You can send requests to any
11871 one of the nodes, and all three will respond as if you are working with
11872 a single CouchDB cluster.
11873 For a proper production setup, you’d now set up an HTTP reverse proxy
11875 in front of the cluster, for load balancing and SSL termination. We
11876 recommend HAProxy, but others can be used. Sample configurations are
11877 available in the best-practices section.
11878 The Cluster Setup API
11880 If you would prefer to manually configure your CouchDB cluster, CouchDB
11881 exposes the _cluster_setup endpoint for that purpose. After installa‐
11882 tion and initial setup/config, we can set up the cluster. On each node
11883 we need to run the following command to set up the node:
11884 curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "node_count":"3"}'
11886 After that we can join all the nodes together. Choose one node as the
11888 “setup coordination node” to run all these commands on. This “setup
11889 coordination node” only manages the setup and requires all other nodes
11890 to be able to see it and vice versa. It has no special purpose beyond
11891 the setup process; CouchDB does not have the concept of a “master” node
11892 in a cluster.
11893 Setup will not work with unavailable nodes. All nodes must be online
11895 and properly preconfigured before the cluster setup process can begin.
11896 To join a node to the cluster, run these commands for each node you
11898 want to add:
11899 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "port": 5984, "node_count": "3", "remote_node": "<remote-node-ip>", "remote_current_user": "<remote-node-username>", "remote_current_password": "<remote-node-password>" }'
11901 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "add_node", "host":"<remote-node-ip>", "port": <remote-node-port>, "username": "admin", "password":"password"}'
11902 This will join the two nodes together. Keep running the above commands
11904 for each node you want to add to the cluster. Once this is done run the
11905 following command to complete the cluster setup and add the system
11906 databases:
11907 curl -X POST -H "Content-Type: application/json" http://admin:password@<setup-coordination-node>:5984/_cluster_setup -d '{"action": "finish_cluster"}'
11909 Verify install:
11911 curl http://admin:password@<setup-coordination-node>:5984/_cluster_setup
11913 Response:
11915 {"state":"cluster_finished"}
11917 Verify all cluster nodes are connected:
11919 curl http://admin:password@<setup-coordination-node>:5984/_membership
11921 Response:
11923 {
11925 "all_nodes": [
11926 "couchdb@couch1.test.com",
11927 "couchdb@couch2.test.com",
11928 "couchdb@couch3.test.com",
11929 ],
11930 "cluster_nodes": [
11931 "couchdb@couch1.test.com",
11932 "couchdb@couch2.test.com",
11933 "couchdb@couch3.test.com",
11934 ]
11935 }
11936 Ensure the all_nodes and cluster_nodes lists match.
11938 You CouchDB cluster is now set up.
11940
11942 Introduction To Configuring
11943 Configuration files
11944 By default, CouchDB reads configuration files from the following loca‐
11945 tions, in the following order:
11946 1. etc/default.ini
11948 2. etc/default.d/*.ini
11950 3. etc/local.ini
11952 4. etc/local.d/*.ini
11954 All paths are specified relative to the CouchDB installation directory:
11956 /opt/couchdb recommended on UNIX-like systems, C:\CouchDB recommended
11957 on Windows systems, and a combination of two directories on macOS:
11958 Applications/Apache CouchDB.app/Contents/Resources/couchdbx-core/etc
11959 for the default.ini and default.d directories, and /Users/you‐
11960 ruser/Library/Application Support/CouchDB2/etc/couchdb for the
11961 local.ini and local.d directories.
11962 Settings in successive documents override the settings in earlier
11964 entries. For example, setting the chttpd/bind_address parameter in
11965 local.ini would override any setting in default.ini.
11966 WARNING:
11968 The default.ini file may be overwritten during an upgrade or
11969 re-installation, so localised changes should be made to the
11970 local.ini file or files within the local.d directory.
11971 The configuration file chain may be changed by setting the ERL_FLAGS
11973 environment variable:
11974 export ERL_FLAGS="-couch_ini /path/to/my/default.ini /path/to/my/local.ini"
11976 or by placing the -couch_ini .. flag directly in the etc/vm.args file.
11978 Passing -couch_ini .. as a command-line argument when launching couchdb
11979 is the same as setting the ERL_FLAGS environment variable.
11980 WARNING:
11982 The environment variable/command-line flag overrides any -couch_ini
11983 option specified in the etc/vm.args file. And, BOTH of these options
11984 completely override CouchDB from searching in the default locations.
11985 Use these options only when necessary, and be sure to track the con‐
11986 tents of etc/default.ini, which may change in future releases.
11987 If there is a need to use different vm.args or sys.config files, for
11989 example, in different locations to the ones provided by CouchDB, or you
11990 don’t want to edit the original files, the default locations may be
11991 changed by setting the COUCHDB_ARGS_FILE or COUCHDB_SYSCONFIG_FILE
11992 environment variables:
11993 export COUCHDB_ARGS_FILE="/path/to/my/vm.args"
11995 export COUCHDB_SYSCONFIG_FILE="/path/to/my/sys.config"
11996 Parameter names and values
11998 All parameter names are case-sensitive. Every parameter takes a value
11999 of one of five types: boolean, integer, string, tuple and proplist.
12000 Boolean values can be written as true or false.
12001 Parameters with value type of tuple or proplist are following the
12003 Erlang requirement for style and naming.
12004 Setting parameters via the configuration file
12006 The common way to set some parameters is to edit the local.ini file
12007 (location explained above).
12008 For example:
12010 ; This is a comment
12012 [section]
12013 param = value ; inline comments are allowed
12014 Each configuration file line may contains section definition, parameter
12016 specification, empty (space and newline characters only) or commented
12017 line. You can set up inline commentaries for sections or parameters.
12018 The section defines group of parameters that are belongs to some spe‐
12020 cific CouchDB subsystem. For instance, httpd section holds not only
12021 HTTP server parameters, but also others that directly interacts with
12022 it.
12023 The parameter specification contains two parts divided by the equal
12025 sign (=): the parameter name on the left side and the parameter value
12026 on the right one. The leading and following whitespace for = is an
12027 optional to improve configuration readability.
12028 NOTE:
12030 In case when you’d like to remove some parameter from the
12031 default.ini without modifying that file, you may override in
12032 local.ini, but without any value:
12033 [compactions]
12035 _default =
12036 This could be read as: “remove the _default parameter from the com‐
12038 pactions section if it was ever set before”.
12039 The semicolon (;) signals the start of a comment. Everything after this
12041 character is ignored by CouchDB.
12042 After editing the configuration file, CouchDB should be restarted to
12044 apply any changes.
12045 Setting parameters via the HTTP API
12047 Alternatively, configuration parameters can be set via the HTTP API.
12048 This API allows changing CouchDB configuration on-the-fly without
12049 requiring a server restart:
12050 curl -X PUT http://localhost:5984/_node/<name@host>/_config/uuids/algorithm -d '"random"'
12052 The old parameter’s value is returned in the response:
12054 "sequential"
12056 You should be careful changing configuration via the HTTP API since
12058 it’s possible to make CouchDB unreachable, for example, by changing
12059 the chttpd/bind_address:
12060 curl -X PUT http://localhost:5984/_node/<name@host>/_config/chttpd/bind_address -d '"10.10.0.128"'
12062 If you make a typo or the specified IP address is not available from
12064 your network, CouchDB will be unreachable. The only way to resolve this
12065 will be to remote into the server, correct the config file, and restart
12066 CouchDB. To protect yourself against such accidents you may set the
12067 httpd/config_whitelist of permitted configuration parameters for
12068 updates via the HTTP API. Once this option is set, further changes to
12069 non-whitelisted parameters must take place via the configuration file,
12070 and in most cases, will also require a server restart before taking
12071 effect.
12072 Configuring the local node
12074 While the HTTP API allows configuring all nodes in the cluster, as a
12075 convenience, you can use the literal string _local in place of the node
12076 name, to interact with the local node’s configuration. For example:
12077 curl -X PUT http://localhost:5984/_node/_local/_config/uuids/algorithm -d '"random"'
12079 Base Configuration
12081 Base CouchDB Options
12082 [couchdb]
12083 attachment_stream_buffer_size
12085 Higher values may result in better read performance due
12086 to fewer read operations and/or more OS page cache hits.
12087 However, they can also increase overall response time for
12088 writes when there are many attachment write requests in
12089 parallel.
12090 [couchdb]
12092 attachment_stream_buffer_size = 4096
12093 database_dir
12095 Specifies location of CouchDB database files (*.couch
12096 named). This location should be writable and readable for
12097 the user the CouchDB service runs as (couchdb by
12098 default).
12099 [couchdb]
12101 database_dir = /var/lib/couchdb
12102 default_security
12104 Changed in version 3.0: admin_only is now the default.
12105 Default security object for databases if not explicitly
12108 set. When set to everyone, anyone can performs reads and
12109 writes. When set to admin_only, only admins can read and
12110 write. When set to admin_local, sharded databases can be
12111 read and written by anyone but the shards can only be
12112 read and written by admins.
12113 [couchdb] default_security = admin_only
12114 enable_database_recovery
12116 Enable this to only “soft-delete” databases when DELETE
12117 /{db} DELETE requests are made. This will place a
12118 .recovery directory in your data directory and move
12119 deleted databases/shards there instead. You can then man‐
12120 ually delete these files later, as desired.
12121 Default is false.
12123 [couchdb]
12125 enable_database_recovery = false
12126 file_compression
12128 Changed in version 1.2: Added Google Snappy compression
12129 algorithm.
12130 Method used to compress everything that is appended to
12133 database and view index files, except for attachments
12134 (see the attachments section). Available methods are:
12135 · none: no compression
12137 · snappy: use Google Snappy, a very fast compres‐
12139 sor/decompressor
12140 · deflate_N: use zlib’s deflate; N is the compression
12142 level which ranges from 1 (fastest, lowest compression
12143 ratio) to 9 (slowest, highest compression ratio)
12144 [couchdb]
12146 file_compression = snappy
12147 maintenance_mode
12149 A CouchDB node may be put into two distinct maintenance
12150 modes by setting this configuration parameter.
12151 · true: The node will not respond to clustered requests
12153 from other nodes and the /_up endpoint will return a
12154 404 response.
12155 · nolb: The /_up endpoint will return a 404 response.
12157 · false: The node responds normally, /_up returns a 200
12159 response.
12160 It is expected that the administrator has configured a
12162 load balancer in front of the CouchDB nodes in the clus‐
12163 ter. This load balancer should use the /_up endpoint to
12164 determine whether or not to send HTTP requests to any
12165 particular node. For HAProxy, the following config is
12166 appropriate:
12167 http-check disable-on-404
12169 option httpchk GET /_up
12170 max_dbs_open
12172 This option places an upper bound on the number of data‐
12173 bases that can be open at once. CouchDB reference counts
12174 database accesses internally and will close idle data‐
12175 bases as needed. Sometimes it is necessary to keep more
12176 than the default open at once, such as in deployments
12177 where many databases will be replicating continuously.
12178 [couchdb]
12180 max_dbs_open = 100
12181 max_document_size
12183 Changed in version 3.0.0.
12184 Limit maximum document body size. Size is calculated
12187 based on the serialized Erlang representation of the JSON
12188 document body, because that reflects more accurately the
12189 amount of storage consumed on disk. In particular, this
12190 limit does not include attachments.
12191 HTTP requests which create or update documents will fail
12193 with error code 413 if one or more documents is larger
12194 than this configuration value.
12195 In case of _update handlers, document size is checked
12197 after the transformation and right before being inserted
12198 into the database.
12199 [couchdb]
12201 max_document_size = 8000000 ; bytes
12202 WARNING:
12204 Before version 2.1.0 this setting was implemented by
12205 simply checking http request body sizes. For individ‐
12206 ual document updates via PUT that approximation was
12207 close enough, however that is not the case for
12208 _bulk_docs endpoint. After 2.1.0 a separate configura‐
12209 tion parameter was defined:
12210 httpd/max_http_request_size, which can be used to
12211 limit maximum http request sizes. After upgrade, it is
12212 advisable to review those settings and adjust them
12213 accordingly.
12214 os_process_timeout
12216 If an external process, such as a query server or exter‐
12217 nal process, runs for this amount of milliseconds without
12218 returning any results, it will be terminated. Keeping
12219 this value smaller ensures you get expedient errors, but
12220 you may want to tweak it for your specific needs.
12221 [couchdb]
12223 os_process_timeout = 5000 ; 5 sec
12224 single_node
12226 New in version 3.0.0.
12227 When this configuration setting is set to true, automati‐
12230 cally create the system databases on startup. Must be set
12231 false for a clustered CouchDB installation.
12232 uri_file
12234 This file contains the full URI that can be used to
12235 access this instance of CouchDB. It is used to help dis‐
12236 cover the port CouchDB is running on (if it was set to 0
12237 (e.g. automatically assigned any free one). This file
12238 should be writable and readable for the user that runs
12239 the CouchDB service (couchdb by default).
12240 [couchdb]
12242 uri_file = /var/run/couchdb/couchdb.uri
12243 users_db_security_editable
12245 New in version 3.0.0.
12246 When this configuration setting is set to false, reject
12249 any attempts to modify the _users database security
12250 object. Modification of this object is deprecated in 3.x
12251 and will be completely disallowed in CouchDB 4.x.
12252 users_db_suffix
12254 Specifies the suffix (last component of a name) of the
12255 system database for storing CouchDB users.
12256 [couchdb]
12258 users_db_suffix = _users
12259 WARNING:
12261 If you change the database name, do not forget to
12262 remove or clean up the old database, since it will no
12263 longer be protected by CouchDB.
12264 util_driver_dir
12266 Specifies location of binary drivers (icu, ejson, etc.).
12267 This location and its contents should be readable for the
12268 user that runs the CouchDB service.
12269 [couchdb]
12271 util_driver_dir = /usr/lib/couchdb/erlang/lib/couch-1.5.0/priv/lib
12272 uuid New in version 1.3.
12274 Unique identifier for this CouchDB server instance.
12277 [couchdb]
12279 uuid = 0a959b9b8227188afc2ac26ccdf345a6
12280 view_index_dir
12282 Specifies location of CouchDB view index files. This
12283 location should be writable and readable for the user
12284 that runs the CouchDB service (couchdb by default).
12285 [couchdb]
12287 view_index_dir = /var/lib/couchdb
12288 Configuring Clustering
12290 Cluster Options
12291 [cluster]
12292 q
12294 Sets the default number of shards for newly created databases.
12296 The default value, 2, splits a database into 2 separate parti‐
12297 tions.
12298 [cluster]
12300 q = 2
12301 For systems with only a few, heavily accessed, large databases,
12303 or for servers with many CPU cores, consider increasing this
12304 value to 4 or 8.
12305 The value of q can also be overridden on a per-DB basis, at DB
12307 creation time.
12308 SEE ALSO:
12310 PUT /{db}
12311 n
12313 Sets the number of replicas of each document in a cluster.
12315 CouchDB will only place one replica per node in a cluster. When
12316 set up through the Cluster Setup Wizard, a standalone single
12317 node will have n = 1, a two node cluster will have n = 2, and
12318 any larger cluster will have n = 3. It is recommended not to set
12319 n greater than 3.
12320 [cluster]
12322 n = 3
12323 placement
12325 WARNING:
12327 Use of this option will override the n option for replica
12328 cardinality. Use with care.
12329 Sets the cluster-wide replica placement policy when creating new
12331 databases. The value must be a comma-delimited list of strings
12332 of the format zone_name:#, where zone_name is a zone as speci‐
12333 fied in the nodes database and # is an integer indicating the
12334 number of replicas to place on nodes with a matching zone_name.
12335 This parameter is not specified by default.
12337 [cluster]
12339 placement = metro-dc-a:2,metro-dc-b:1
12340 SEE ALSO:
12342 cluster/databases/placement
12343 seedlist
12345 An optional, comma-delimited list of node names that this node
12347 should contact in order to join a cluster. If a seedlist is con‐
12348 figured the _up endpoint will return a 404 until the node has
12349 successfully contacted at least one of the members of the
12350 seedlist and replicated an up-to-date copy of the _nodes, _dbs,
12351 and _users system databases.
12352 [cluster] seedlist =
12353 couchdb@node1.example.com,couchdb@node2.example.com
12354 RPC Performance Tuning
12356 [rexi] CouchDB uses distributed Erlang to communicate between nodes in
12357 a cluster. The rexi library provides an optimized RPC mechanism
12358 over this communication channel. There are a few configuration
12359 knobs for this system, although in general the defaults work
12360 well.
12361 buffer_count
12363 The local RPC server will buffer messages if a remote node goes
12365 unavailable. This flag determines how many messages will be
12366 buffered before the local server starts dropping messages.
12367 Default value is 2000.
12368 server_per_node
12370 By default, rexi will spawn one local gen_server process for
12372 each node in the cluster. Disabling this flag will cause CouchDB
12373 to use a single process for all RPC communication, which is not
12374 recommended in high throughput deployments.
12375 stream_limit
12377 New in version 3.0.
12378 This flag comes into play during streaming operations like views
12381 and change feeds. It controls how many messages a remote worker
12382 process can send to a coordinator without waiting for an
12383 acknowledgement from the coordinator process. If this value is
12384 too large the coordinator can become overwhelmed by messages
12385 from the worker processes and actually deliver lower overall
12386 throughput to the client. In CouchDB 2.x this value was
12387 hard-coded to 10. In the 3.x series it is configurable and
12388 defaults to 5. Databases with a high q value are especially
12389 sensitive to this setting.
12390 couch_peruser
12392 couch_peruser Options
12393 [couch_peruser]
12394 enable
12396 If set to true, couch_peruser ensures that a private per-user
12398 database exists for each document in _users. These databases are
12399 writable only by the corresponding user. Databases are in the
12400 following form: userdb-{hex encoded username}.
12401 [couch_peruser]
12403 enable = false
12404 NOTE:
12406 The _users database must exist before couch_peruser can be
12407 enabled.
12408 delete_dbs
12410 If set to true and a user is deleted, the respective database
12412 gets deleted as well.
12413 [couch_peruser]
12415 delete_dbs = false
12416 CouchDB HTTP Server
12418 HTTP Server Options
12419 [chttpd]
12420 NOTE:
12422 In CouchDB 2.x, the chttpd section refers to the standard, clustered
12423 port. All use of CouchDB, aside from a few specific maintenance
12424 tasks as described in this documentation, should be performed over
12425 this port.
12426 bind_address
12428 Defines the IP address by which the clustered port is avail‐
12429 able:
12430 [chttpd]
12432 bind_address = 127.0.0.1
12433 To let CouchDB listen any available IP address, use 0.0.0.0:
12435 [chttpd]
12437 bind_address = 0.0.0.0
12438 For IPv6 support you need to set ::1 if you want to let
12440 CouchDB listen correctly:
12441 [chttpd]
12443 bind_address = ::1
12444 or :: for any available:
12446 [chttpd]
12448 bind_address = ::
12449 port Defines the port number to listen:
12451 [chttpd]
12453 port = 5984
12454 To let CouchDB use any free port, set this option to 0:
12456 [chttpd]
12458 port = 0
12459 prefer_minimal
12461 If a request has the header “Prefer”: “return=minimal”,
12462 CouchDB will only send the headers that are listed for the
12463 prefer_minimal configuration.:
12464 [chttpd]
12466 prefer_minimal = Cache-Control, Content-Length, Content-Range, Content-Type, ETag, Server, Transfer-Encoding, Vary
12467 WARNING:
12469 Removing the Server header from the settings will mean
12470 that the CouchDB server header is replaced with the
12471 MochiWeb server header.
12472 authentication_handlers
12474 List of authentication handlers used by CouchDB. You may
12475 extend them via third-party plugins or remove some of them if
12476 you won’t let users to use one of provided methods:
12477 [chttpd]
12479 authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, default_authentication_handler}
12480 · {chttpd_auth, cookie_authentication_handler}: used for
12482 Cookie auth;
12483 · {chttpd_auth, proxy_authentication_handler}: used for Proxy
12485 auth;
12486 · {chttpd_auth, default_authentication_handler}: used for
12488 Basic auth;
12489 · {couch_httpd_auth, null_authentication_handler}: disables
12491 auth, breaks CouchDB.
12492 [httpd]
12494 allow_jsonp
12496 The true value of this option enables JSONP support (it’s
12497 false by default):
12498 [httpd]
12500 allow_jsonp = false
12501 changes_timeout
12503 Specifies default timeout value for Changes Feed in mil‐
12504 liseconds (60000 by default):
12505 [httpd]
12507 changes_timeout = 60000 ; 60 seconds
12508 config_whitelist
12510 Sets the configuration modification whitelist. Only
12511 whitelisted values may be changed via the config API. To
12512 allow the admin to change this value over HTTP, remember
12513 to include {httpd,config_whitelist} itself. Excluding it
12514 from the list would require editing this file to update
12515 the whitelist:
12516 [httpd]
12518 config_whitelist = [{httpd,config_whitelist}, {log,level}, {etc,etc}]
12519 enable_cors
12521 New in version 1.3.
12522 Controls CORS feature:
12525 [httpd]
12527 enable_cors = false
12528 server_options
12530 Server options for the MochiWeb component of CouchDB can
12531 be added to the configuration files:
12532 [httpd]
12534 server_options = [{backlog, 128}, {acceptor_pool_size, 16}]
12535 The options supported are a subset of full options sup‐
12537 ported by the TCP/IP stack. A list of the supported
12538 options are provided in the Erlang inet documentation.
12539 secure_rewrites
12541 This option allow to isolate databases via subdomains:
12542 [httpd]
12544 secure_rewrites = true
12545 socket_options
12547 The socket options for the listening socket in CouchDB,
12548 as set at the beginning of ever request, can be specified
12549 as a list of tuples. For example:
12550 [httpd]
12552 socket_options = [{sndbuf, 262144}]
12553 The options supported are a subset of full options sup‐
12555 ported by the TCP/IP stack. A list of the supported
12556 options are provided in the Erlang inet documentation.
12557 x_forwarded_host
12559 The x_forwarded_host header (X-Forwarded-Host by default)
12560 is used to forward the original value of the Host header
12561 field in case, for example, if a reverse proxy is rewrit‐
12562 ing the “Host” header field to some internal host name
12563 before forward the request to CouchDB:
12564 [httpd]
12566 x_forwarded_host = X-Forwarded-Host
12567 This header has higher priority above Host one, if only
12569 it exists in the request.
12570 x_forwarded_proto
12572 x_forwarded_proto header (X-Forwarder-Proto by default)
12573 is used for identifying the originating protocol of an
12574 HTTP request, since a reverse proxy may communicate with
12575 CouchDB instance using HTTP even if the request to the
12576 reverse proxy is HTTPS:
12577 [httpd]
12579 x_forwarded_proto = X-Forwarded-Proto
12580 x_forwarded_ssl
12582 The x_forwarded_ssl header (X-Forwarded-Ssl by default)
12583 tells CouchDB that it should use the https scheme instead
12584 of the http. Actually, it’s a synonym for X-For‐
12585 warded-Proto: https header, but used by some reverse
12586 proxies:
12587 [httpd]
12589 x_forwarded_ssl = X-Forwarded-Ssl
12590 enable_xframe_options
12592 Controls Enables or disabled feature:
12593 [httpd]
12595 enable_xframe_options = false
12596 max_http_request_size
12598 Changed in version 2.1.0.
12599 Limit the maximum size of the HTTP request body. This
12602 setting applies to all requests and it doesn’t discrimi‐
12603 nate between single vs. multi-document operations. So
12604 setting it to 1MB would block a PUT of a document larger
12605 than 1MB, but it might also block a _bulk_docs update of
12606 1000 1KB documents, or a multipart/related update of a
12607 small document followed by two 512KB attachments. This
12608 setting is intended to be used as a protection against
12609 maliciously large HTTP requests rather than for limiting
12610 maximum document sizes.
12611 [httpd]
12613 max_http_request_size = 4294967296 ; 4 GB
12614 WARNING:
12616 Before version 2.1.0 couchdb/max_document_size was
12617 implemented effectively as max_http_request_size. That
12618 is, it checked HTTP request bodies instead of document
12619 sizes. After the upgrade, it is advisable to review
12620 the usage of these configuration settings.
12621 HTTPS (SSL/TLS) Options
12623 [ssl] CouchDB supports TLS/SSL natively, without the use of a proxy
12624 server.
12625 HTTPS setup can be tricky, but the configuration in CouchDB was
12627 designed to be as easy as possible. All you need is two files; a
12628 certificate and a private key. If you have an official certifi‐
12629 cate from a certificate authority, both should be in your pos‐
12630 session already.
12631 If you just want to try this out and don’t want to go through
12633 the hassle of obtaining an official certificate, you can create
12634 a self-signed certificate. Everything will work the same, but
12635 clients will get a warning about an insecure certificate.
12636 You will need the OpenSSL command line tool installed. It proba‐
12638 bly already is.
12639 shell> mkdir /etc/couchdb/cert
12641 shell> cd /etc/couchdb/cert
12642 shell> openssl genrsa > privkey.pem
12643 shell> openssl req -new -x509 -key privkey.pem -out couchdb.pem -days 1095
12644 shell> chmod 600 privkey.pem couchdb.pem
12645 shell> chown couchdb privkey.pem couchdb.pem
12646 Now, you need to edit CouchDB’s configuration, by editing your
12648 local.ini file. Here is what you need to do.
12649 Under the [ssl] section, enable HTTPS and set up the newly gen‐
12651 erated certificates:
12652 [ssl]
12654 enable = true
12655 cert_file = /etc/couchdb/cert/couchdb.pem
12656 key_file = /etc/couchdb/cert/privkey.pem
12657 For more information please read certificates HOWTO.
12659 Now start (or restart) CouchDB. You should be able to connect to
12661 it using HTTPS on port 6984:
12662 shell> curl https://127.0.0.1:6984/
12664 curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
12665 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
12666 More details here: http://curl.haxx.se/docs/sslcerts.html
12667 curl performs SSL certificate verification by default, using a "bundle"
12669 of Certificate Authority (CA) public keys (CA certs). If the default
12670 bundle file isn't adequate, you can specify an alternate file
12671 using the --cacert option.
12672 If this HTTPS server uses a certificate signed by a CA represented in
12673 the bundle, the certificate verification probably failed due to a
12674 problem with the certificate (it might be expired, or the name might
12675 not match the domain name in the URL).
12676 If you'd like to turn off curl's verification of the certificate, use
12677 the -k (or --insecure) option.
12678 Oh no! What happened?! Remember, clients will notify their users
12680 that your certificate is self signed. curl is the client in this
12681 case and it notifies you. Luckily you trust yourself (don’t
12682 you?) and you can specify the -k option as the message reads:
12683 shell> curl -k https://127.0.0.1:6984/
12685 {"couchdb":"Welcome","version":"1.5.0"}
12686 All done.
12688 For performance reasons, and for ease of setup, you may still
12690 wish to terminate HTTPS connections at your load balancer /
12691 reverse proxy, then use unencrypted HTTP between it and your
12692 CouchDB cluster. This is a recommended approach.
12693 cacert_file
12695 The path to a file containing PEM encoded CA certifi‐
12696 cates. The CA certificates are used to build the server
12697 certificate chain, and for client authentication. Also
12698 the CAs are used in the list of acceptable client CAs
12699 passed to the client when a certificate is requested. May
12700 be omitted if there is no need to verify the client and
12701 if there are not any intermediate CAs for the server cer‐
12702 tificate:
12703 [ssl]
12705 cacert_file = /etc/ssl/certs/ca-certificates.crt
12706 cert_file
12708 Path to a file containing the user’s certificate:
12709 [ssl]
12711 cert_file = /etc/couchdb/cert/couchdb.pem
12712 key_file
12714 Path to file containing user’s private PEM encoded key:
12715 [ssl]
12717 key_file = /etc/couchdb/cert/privkey.pem
12718 password
12720 String containing the user’s password. Only used if the
12721 private key file is password protected:
12722 [ssl]
12724 password = somepassword
12725 ssl_certificate_max_depth
12727 Maximum peer certificate depth (must be set even if cer‐
12728 tificate validation is off):
12729 [ssl]
12731 ssl_certificate_max_depth = 1
12732 verify_fun
12734 The verification fun (optional) if not specified, the
12735 default verification fun will be used:
12736 [ssl]
12738 verify_fun = {Module, VerifyFun}
12739 verify_ssl_certificates
12741 Set to true to validate peer certificates:
12742 [ssl]
12744 verify_ssl_certificates = false
12745 fail_if_no_peer_cert
12747 Set to true to terminate the TLS/SSL handshake with a
12748 handshake_failure alert message if the client does not
12749 send a certificate. Only used if verify_ssl_certificates
12750 is true. If set to false it will only fail if the client
12751 sends an invalid certificate (an empty certificate is
12752 considered valid):
12753 [ssl]
12755 fail_if_no_peer_cert = false
12756 secure_renegotiate
12758 Set to true to reject renegotiation attempt that does not
12759 live up to RFC 5746:
12760 [ssl]
12762 secure_renegotiate = true
12763 ciphers
12765 Set to the cipher suites that should be supported which
12766 can be specified in erlang format
12767 “{ecdhe_ecdsa,aes_128_cbc,sha256}” or in OpenSSL format
12768 “ECDHE-ECDSA-AES128-SHA256”.
12769 [ssl]
12771 ciphers = ["ECDHE-ECDSA-AES128-SHA256", "ECDHE-ECDSA-AES128-SHA"]
12772 tls_versions
12774 Set to a list of permitted SSL/TLS protocol versions:
12775 [ssl]
12777 tls_versions = [tlsv1 | 'tlsv1.1' | 'tlsv1.2']
12778 Cross-Origin Resource Sharing
12780 [cors] New in version 1.3: added CORS support, see JIRA COUCHDB-431
12781 CORS, or “Cross-Origin Resource Sharing”, allows a resource such
12784 as a web page running JavaScript inside a browser, to make AJAX
12785 requests (XMLHttpRequests) to a different domain, without com‐
12786 promising the security of either party.
12787 A typical use case is to have a static website hosted on a CDN
12789 make requests to another resource, such as a hosted CouchDB
12790 instance. This avoids needing an intermediary proxy, using JSONP
12791 or similar workarounds to retrieve and host content.
12792 While CouchDB’s integrated HTTP server has support for document
12794 attachments makes this less of a constraint for pure CouchDB
12795 projects, there are many cases where separating the static con‐
12796 tent from the database access is desirable, and CORS makes this
12797 very straightforward.
12798 By supporting CORS functionality, a CouchDB instance can accept
12800 direct connections to protected databases and instances, without
12801 the browser functionality being blocked due to same-origin con‐
12802 straints. CORS is supported today on over 90% of recent
12803 browsers.
12804 CORS support is provided as experimental functionality in 1.3,
12806 and as such will need to be enabled specifically in CouchDB’s
12807 configuration. While all origins are forbidden from making
12808 requests by default, support is available for simple requests,
12809 preflight requests and per-vhost configuration.
12810 This section requires httpd/enable_cors option have true value:
12812 [httpd]
12814 enable_cors = true
12815 credentials
12817 By default, neither authentication headers nor cookies
12818 are included in requests and responses. To do so requires
12819 both setting XmlHttpRequest.withCredentials = true on the
12820 request object in the browser and enabling credentials
12821 support in CouchDB.
12822 [cors]
12824 credentials = true
12825 CouchDB will respond to a credentials-enabled CORS
12827 request with an additional header, Access-Con‐
12828 trol-Allow-Credentials=true.
12829 origins
12831 List of origins separated by a comma, * means accept all.
12832 You can’t set origins = * and credentials = true option
12833 at the same time:
12834 [cors]
12836 origins = *
12837 Access can be restricted by protocol, host and optionally
12839 by port. Origins must follow the scheme:
12840 http://example.com:80:
12841 [cors]
12843 origins = http://localhost, https://localhost, http://couch.mydev.name:8080
12844 Note that by default, no origins are accepted. You must
12846 define them explicitly.
12847 headers
12849 List of accepted headers separated by a comma:
12850 [cors]
12852 headers = X-Couch-Id, X-Couch-Rev
12853 methods
12855 List of accepted methods:
12856 [cors]
12858 methods = GET,POST
12859 max_age
12861 Sets the Access-Control-Max-Age header in seconds. Use it
12862 to avoid repeated OPTIONS requests.
12863 [cors] max_age = 3600
12864 SEE ALSO:
12866 Original JIRA implementation ticket
12867 Standards and References:
12869 · IETF RFCs relating to methods: RFC 2618, RFC 2817, RFC 5789
12871 · IETF RFC for Web Origins: RFC 6454
12873 · W3C CORS standard
12875 Mozilla Developer Network Resources:
12877 · Same origin policy for URIs
12879 · HTTP Access Control
12881 · Server-side Access Control
12883 · JavaScript same origin policy
12885 Client-side CORS support and usage:
12887 · CORS browser support matrix
12889 · COS tutorial
12891 · XHR with CORS
12893 Per Virtual Host Configuration
12895 WARNING:
12896 Virtual Hosts are deprecated in CouchDB 3.0, and will be removed in
12897 CouchDB 4.0.
12898 To set the options for a vhosts, you will need to create a section with
12900 the vhost name prefixed by cors:. Example case for the vhost exam‐
12901 ple.com:
12902 [cors:example.com]
12904 credentials = false
12905 ; List of origins separated by a comma
12906 origins = *
12907 ; List of accepted headers separated by a comma
12908 headers = X-CouchDB-Header
12909 ; List of accepted methods
12910 methods = HEAD, GET
12911 A video from 2010 on vhost and rewrite configuration is available, but
12913 is not guaranteed to match current syntax or behaviour.
12914 Virtual Hosts
12916 WARNING:
12917 Virtual Hosts are deprecated in CouchDB 3.0, and will be removed in
12918 CouchDB 4.0.
12919 [vhosts]
12921 CouchDB can map requests to different locations based on the
12922 Host header, even if they arrive on the same inbound IP address.
12923 This allows different virtual hosts on the same machine to map
12925 to different databases or design documents, etc. The most common
12926 use case is to map a virtual host to a Rewrite Handler, to pro‐
12927 vide full control over the application’s URIs.
12928 To add a virtual host, add a CNAME pointer to the DNS for your
12930 domain name. For development and testing, it is sufficient to
12931 add an entry in the hosts file, typically /etc/hosts` on
12932 Unix-like operating systems:
12933 # CouchDB vhost definitions, refer to local.ini for further details
12935 127.0.0.1 couchdb.local
12936 Test that this is working:
12938 $ ping -n 2 couchdb.local
12940 PING couchdb.local (127.0.0.1) 56(84) bytes of data.
12941 64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.025 ms
12942 64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.051 ms
12943 Finally, add an entry to your configuration file in the [vhosts]
12945 section:
12946 [vhosts]
12948 couchdb.local:5984 = /example
12949 *.couchdb.local:5984 = /example
12950 If your CouchDB is listening on the the default HTTP port (80),
12952 or is sitting behind a proxy, then you don’t need to specify a
12953 port number in the vhost key.
12954 The first line will rewrite the request to display the content
12956 of the example database. This rule works only if the Host header
12957 is couchdb.local and won’t work for CNAMEs. The second rule, on
12958 the other hand, matches all CNAMEs to example db, so that both
12959 www.couchdb.local and db.couchdb.local will work.
12960 Rewriting Hosts to a Path
12962 Like in the _rewrite handler you can match some variable and use them
12963 to create the target path. Some examples:
12964 [vhosts]
12966 *.couchdb.local = /*
12967 :dbname. = /:dbname
12968 :ddocname.:dbname.example.com = /:dbname/_design/:ddocname/_rewrite
12969 The first rule passes the wildcard as dbname. The second one does the
12971 same, but uses a variable name. And the third one allows you to use any
12972 URL with ddocname in any database with dbname.
12973 X-Frame-Options
12975 X-Frame-Options is a response header that controls whether a http
12976 response can be embedded in a <frame>, <iframe> or <object>. This is a
12977 security feature to help against clickjacking.
12978 [x_frame_options] ; Settings same-origin will return
12979 X-Frame-Options: SAMEORIGIN. ; If same origin is set, it will
12980 ignore the hosts setting ; same_origin = true ; Settings hosts will
12981 ; return X-Frame-Options: ALLOW-FROM https://example.com/ ; List of
12982 hosts separated by a comma. * means accept all ; hosts =
12983 If xframe_options is enabled it will return X-Frame-Options: DENY by
12985 default. If same_origin is enabled it will return X-Frame-Options:
12986 SAMEORIGIN. A X-FRAME-OPTIONS: ALLOW-FROM url will be returned when
12987 same_origin is false, and the HOST header matches one of the urls in
12988 the hosts config. Otherwise a X-Frame-Options: DENY will be returned.
12989 Authentication and Authorization
12991 Server Administrators
12992 [admins]
12993 Changed in version 3.0.0: CouchDB requires an admin account to start.
12995 If an admin account has not been created, CouchDB will print an error
12996 message and terminate.
12997 CouchDB server administrators and passwords are not stored in the
12999 _users database, but in the last [admins] section that CouchDB finds
13000 when loading its ini files. See :config:intro for details on config
13001 file order and behaviour. This file (which could be something like
13002 etc/local.ini or etc/local.d/10-admins.ini on a Debian/Ubuntu system
13003 installed from packages) should be appropriately secured and readable
13004 only by system administrators:
13005 [admins]
13007 ;admin = mysecretpassword
13008 admin = -hashed-6d3c30241ba0aaa4e16c6ea99224f915687ed8cd,7f4a3e05e0cbc6f48a0035e3508eef90
13009 architect = -pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000
13010 Administrators can be added directly to the [admins] section, and when
13012 CouchDB is restarted, the passwords will be salted and encrypted. You
13013 may also use the HTTP interface to create administrator accounts; this
13014 way, you don’t need to restart CouchDB, and there’s no need to tempo‐
13015 rarily store or transmit passwords in plaintext. The HTTP
13016 /_node/{node-name}/_config/admins endpoint supports querying, deleting
13017 or creating new admin accounts:
13018 GET /_node/nonode@nohost/_config/admins HTTP/1.1
13020 Accept: application/json
13021 Host: localhost:5984
13022 HTTP/1.1 200 OK
13024 Cache-Control: must-revalidate
13025 Content-Length: 196
13026 Content-Type: application/json
13027 Date: Fri, 30 Nov 2012 11:37:18 GMT
13028 Server: CouchDB (Erlang/OTP)
13029 {
13031 "admin": "-hashed-6d3c30241ba0aaa4e16c6ea99224f915687ed8cd,7f4a3e05e0cbc6f48a0035e3508eef90",
13032 "architect": "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13033 }
13034 If you already have a salted, encrypted password string (for example,
13036 from an old ini file, or from a different CouchDB server), then you can
13037 store the “raw” encrypted string, without having CouchDB doubly encrypt
13038 it.
13039 PUT /_node/nonode@nohost/_config/admins/architect?raw=true HTTP/1.1
13041 Accept: application/json
13042 Content-Type: application/json
13043 Content-Length: 89
13044 Host: localhost:5984
13045 "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13047 HTTP/1.1 200 OK
13049 Cache-Control: must-revalidate
13050 Content-Length: 89
13051 Content-Type: application/json
13052 Date: Fri, 30 Nov 2012 11:39:18 GMT
13053 Server: CouchDB (Erlang/OTP)
13054 "-pbkdf2-43ecbd256a70a3a2f7de40d2374b6c3002918834,921a12f74df0c1052b3e562a23cd227f,10000"
13056 Further details are available in security, including configuring the
13058 work factor for PBKDF2, and the algorithm itself at PBKDF2 (RFC-2898).
13059 Changed in version 1.4: PBKDF2 server-side hashed salted password sup‐
13061 port added, now as a synchronous call for the _config/admins API.
13062 Authentication Configuration
13066 [chttpd]
13067 require_valid_user
13069 When this option is set to true, no requests are allowed
13070 from anonymous users. Everyone must be authenticated.
13071 [chttpd]
13073 require_valid_user = false
13074 require_valid_user_except_for_up
13076 When this option is set to true, no requests are allowed
13077 from anonymous users, except for the /_up endpoint.
13078 Everyone else must be authenticated.
13079 [chttpd]
13081 require_valid_user_except_for_up = false
13082 [couch_httpd_auth]
13084 allow_persistent_cookies
13086 When set to true, CouchDB will refresh the session cookie
13087 whenever the session is nearing expiration.
13088 [couch_httpd_auth]
13090 allow_persistent_cookies = true
13091 cookie_domain
13093 New in version 2.1.1.
13094 Configures the domain attribute of the AuthSession
13097 cookie. By default the domain attribute is empty, result‐
13098 ing in the cookie being set on CouchDB’s domain.
13099 [couch_httpd_auth]
13101 cookie_domain = example.com
13102 same_site
13104 New in version 3.0.0.
13105 When this option is set to a non-empty value, a SameSite
13108 attribute is added to the AuthSession cookie. Valid val‐
13109 ues are none, lax or strict.:
13110 [couch_httpd_auth]
13112 same_site = strict
13113 auth_cache_size
13115 Number of userctx_object to cache in memory, to reduce
13116 disk lookups.
13117 [couch_httpd_auth]
13119 auth_cache_size = 50
13120 authentication_redirect
13122 Specifies the location for redirection on successful
13123 authentication if a text/html response is accepted by the
13124 client (via an Accept header).
13125 [couch_httpd_auth]
13127 authentication_redirect = /_utils/session.html
13128 iterations
13130 New in version 1.3.
13131 The number of iterations for password hashing by the
13134 PBKDF2 algorithm. A higher number provides better hash
13135 durability, but comes at a cost in performance for each
13136 request that requires authentication.
13137 [couch_httpd_auth]
13139 iterations = 10000
13140 min_iterations
13142 New in version 1.6.
13143 The minimum number of iterations allowed for passwords
13146 hashed by the PBKDF2 algorithm. Any user with fewer iter‐
13147 ations is forbidden.
13148 [couch_httpd_auth]
13150 min_iterations = 100
13151 max_iterations
13153 New in version 1.6.
13154 The maximum number of iterations allowed for passwords
13157 hashed by the PBKDF2 algorithm. Any user with greater
13158 iterations is forbidden.
13159 [couch_httpd_auth]
13161 max_iterations = 100000
13162 proxy_use_secret
13164 When this option is set to true, the
13165 couch_httpd_auth/secret option is required for
13166 api/auth/proxy.
13167 [couch_httpd_auth]
13169 proxy_use_secret = false
13170 public_fields
13172 New in version 1.4.
13173 A comma-separated list of field names in user documents
13176 (in couchdb/users_db_suffix) that can be read by any
13177 user. If unset or not specified, authenticated users can
13178 only retrieve their own document.
13179 [couch_httpd_auth]
13181 public_fields = first_name, last_name, contacts, url
13182 NOTE:
13184 Using the public_fields whitelist for user document
13185 properties requires setting the
13186 couch_httpd_auth/users_db_public option to true (the
13187 latter option has no other purpose):
13188 [couch_httpd_auth]
13190 users_db_public = true
13191 require_valid_user
13193 When this option is set to true, no requests are allowed
13194 from anonymous users. Everyone must be authenticated.
13195 [couch_httpd_auth]
13197 require_valid_user = false
13198 secret The secret token is used for api/auth/proxy and for
13200 api/auth/cookie.
13201 [couch_httpd_auth]
13203 secret = 92de07df7e7a3fe14808cef90a7cc0d91
13204 timeout
13206 Number of seconds since the last request before sessions
13207 will be expired.
13208 [couch_httpd_auth]
13210 timeout = 600
13211 users_db_public
13213 New in version 1.4.
13214 Allow all users to view user documents. By default, only
13217 admins may browse all users documents, while users may
13218 browse only their own document.
13219 [couch_httpd_auth]
13221 users_db_public = false
13222 x_auth_roles
13224 The HTTP header name (X-Auth-CouchDB-Roles by default)
13225 that contains the list of a user’s roles, separated by a
13226 comma. Used for api/auth/proxy.
13227 [couch_httpd_auth]
13229 x_auth_roles = X-Auth-CouchDB-Roles
13230 x_auth_token
13232 The HTTP header name (X-Auth-CouchDB-Token by default)
13233 containing the token used to authenticate the authoriza‐
13234 tion. This token is an HMAC-SHA1 created from the
13235 couch_httpd_auth/secret and
13236 couch_httpd_auth/x_auth_username. The secret key should
13237 be the same on the client and the CouchDB node. This
13238 token is optional if the value of the
13239 couch_httpd_auth/proxy_use_secret option is not true.
13240 Used for api/auth/proxy.
13241 [couch_httpd_auth]
13243 x_auth_token = X-Auth-CouchDB-Token
13244 x_auth_username
13246 The HTTP header name (X-Auth-CouchDB-UserName by default)
13247 containing the username. Used for api/auth/proxy.
13248 [couch_httpd_auth]
13250 x_auth_username = X-Auth-CouchDB-UserName
13251 Compaction
13253 Database Compaction Options
13254 [database_compaction]
13255 doc_buffer_size
13257 Specifies the copy buffer’s maximum size in bytes:
13258 [database_compaction]
13260 doc_buffer_size = 524288
13261 checkpoint_after
13263 Triggers a checkpoint after the specified amount of bytes
13264 were successfully copied to the compacted database:
13265 [database_compaction]
13267 checkpoint_after = 5242880
13268 View Compaction Options
13270 [view_compaction]
13271 keyvalue_buffer_size
13273 Specifies maximum copy buffer size in bytes used during
13274 compaction:
13275 [view_compaction]
13277 keyvalue_buffer_size = 2097152
13278 Compaction Daemon
13280 CouchDB ships with an automated, event-driven daemon internally known
13281 as “smoosh” that continuously re-prioritizes the database and secondary
13282 index files on each node and automatically compacts the files that will
13283 recover the most free space according to the following parameters.
13284 [smoosh]
13286 db_channels
13288 A comma-delimited list of channels that are sent the
13289 names of database files when those files are updated.
13290 Each channel can choose whether to enqueue the database
13291 for compaction; once a channel has enqueued the database,
13292 no additional channel in the list will be given the
13293 opportunity to do so.
13294 view_channels
13296 A comma-delimited list of channels that are sent the
13297 names of secondary index files when those files are
13298 updated. Each channel can choose whether to enqueue the
13299 index for compaction; once a channel has enqueued the
13300 index, no additional channel in the list will be given
13301 the opportunity to do so.
13302 staleness
13304 The number of minutes that the (expensive) priority cal‐
13305 culation on an individual can be stale for before it is
13306 recalculated. Defaults to 5.
13307 cleanup_index_files
13309 If set to true, the compaction daemon will delete the
13310 files for indexes that are no longer associated with any
13311 design document. Defaults to false and probably shouldn’t
13312 be changed unless the node is running low on disk space,
13313 and only after considering the ramifications.
13314 wait_secs
13316 The time a channel waits before starting compactions to
13317 allow time to observe the system and make a smarter deci‐
13318 sion about what to compact first. Hardly ever changed
13319 from the default of 30 (seconds).
13320 [smoosh.<channel>]
13322 The following settings control the resource allocation for a given com‐
13324 paction channel.
13325 capacity
13327 The maximum number of items the channel can hold (lowest pri‐
13328 ority item is removed to make room for new items). Defaults
13329 to 9999.
13330 concurrency
13332 The maximum number of jobs that can run concurrently in this
13333 channel. Defaults to 1.
13334 There are also several settings that collectively control whether a
13336 channel will enqueue a file for compaction and how it prioritizes files
13337 within its queue:
13338 max_priority
13340 Each item must have a priority lower than this to be
13341 enqueued. Defaults to infinity.
13342 max_size
13344 The item must be no larger than this many bytes in length to
13345 be enqueued. Defaults to infinity.
13346 min_priority
13348 The item must have a priority at least this high to be
13349 enqueued. Defaults to 5.0 for ratio and 16 MB for slack.
13350 min_changes
13352 The minimum number of changes since last compaction before
13353 the item will be enqueued. Defaults to 0. Currently only
13354 works for databases.
13355 min_size
13357 The item must be at least this many bytes in length to be
13358 enqueued. Defaults to 1mb (1048576 bytes).
13359 priority
13361 The method used to calculate priority. Can be ratio (calcu‐
13362 lated as sizes.file/sizes.active) or slack (calculated as
13363 sizes.file - sizes.active). Defaults to ratio.
13364 Background Indexing
13366 Secondary indexes in CouchDB are not updated during document write
13367 operations. In order to avoid high latencies when reading indexes fol‐
13368 lowing a large block of writes, CouchDB automatically kicks off back‐
13369 ground jobs to keep secondary indexes “warm”. The daemon responsible
13370 for this process is internally known as “ken” and can be configured
13371 using the following settings.
13372 [ken]
13374 batch_channels
13376 This setting controls the number of background view
13377 builds that can be running in parallel at any given time.
13378 The default is 20.
13379 incremental_channels
13381 It is possible for all the slots in the normal build sys‐
13382 tem to be occupied by long-running index rebuilds (e.g.
13383 if new design documents are posted to several databases
13384 simultaneously). In order to avoid already-built indexes
13385 from falling behind when this occurs, CouchDB will allow
13386 for a number of short background indexing jobs to run
13387 even when all slots are full. This setting controls how
13388 many additional short jobs are allowed to run concur‐
13389 rently with the main jobs. The default is 80.
13390 max_incremental_updates
13392 CouchDB estimates whether an indexing job is “incremen‐
13393 tal” or not by looking at the difference in sequence num‐
13394 bers between the current index and the main database. If
13395 the difference is larger than the threshold defined here
13396 the background job will only be allowed to run in the
13397 main queue. Defaults to 1000.
13398 [ken.ignore]
13400 Entries in this configuration section can be used to tell the back‐
13402 ground indexer to skip over specific database shard files. The key must
13403 be the exact name of the shard with the .couch suffix omitted, for
13404 example:
13405 [ken.ignore]
13407 shards/00000000-1fffffff/mydb.1567719095 = true
13408 IO Queue
13410 CouchDB has an internal subsystem that can prioritize IO associated
13411 with certain classes of operations. This subsystem can be configured to
13412 limit the resources devoted to background operations like internal
13413 replication and compaction according to the settings described below.
13414 [ioq]
13416 concurrency
13418 Specifies the maximum number of concurrent in-flight IO
13419 requests that the queueing system will submit:
13420 [ioq]
13422 concurrency = 10
13423 ratio The fraction of the time that a background IO request
13425 will be selected over an interactive IO request when both
13426 queues are non-empty:
13427 [ioq]
13429 ratio = 0.01
13430 [ioq.bypass]
13432 System administrators can choose to submit specific classes of
13433 IO directly to the underlying file descriptor or OS process,
13434 bypassing the queues altogether. Installing a bypass can yield
13435 higher throughput and lower latency, but relinquishes some con‐
13436 trol over prioritization. The following classes are recognized:
13437 os_process
13439 Messages on their way to an external process (e.g.,
13440 couchjs).
13441 read Disk IO fulfilling interactive read requests.
13443 write Disk IO required to update a database.
13445 view_update
13447 Disk IO required to update views and other secondary
13448 indexes.
13449 shard_sync
13451 Disk IO issued by the background replication processes
13452 that fix any inconsistencies between shard copies.
13453 compaction
13455 Disk IO issued by compaction jobs.
13456 Without any configuration CouchDB will enqueue all classes of
13458 IO. The default.ini configuration file that ships with CouchDB
13459 activates a bypass for each of the interactive IO classes and
13460 only background IO goes into the queueing system:
13461 [ioq.bypass]
13463 os_process = true
13464 read = true
13465 write = true
13466 view_update = true
13467 shard_sync = false
13468 compaction = false
13469 Recommendations
13471 The default configuration protects against excessive IO from background
13472 operations like compaction disrupting the latency of interactive opera‐
13473 tions, while maximizing the overall IO throughput devoted to those
13474 interactive requests. There are certain situations where this configu‐
13475 ration could be sub-optimal:
13476 · An administrator may want to devote a larger portion of the overall
13478 IO bandwidth to compaction in order to stay ahead of the incoming
13479 write load. In this it may be necessary to disable the bypass for
13480 write (to help with database compaction) and/or view_update (to help
13481 with view index compaction) and then increase the ratio to give com‐
13482 paction a higher priority.
13483 · A server with a large number of views that do not need to be
13485 comlpetely up-to-date may benefit from removing the bypass on
13486 view_update in order to optimize the latency for regular document
13487 read and write operations, and build the views during quieter peri‐
13488 ods.
13489 Logging
13491 Logging options
13492 [log] CouchDB logging configuration.
13493 writer Current writers include:
13495 · stderr: Logs are sent to stderr.
13497 · file: Logs are sent to the file set in log file.
13499 · syslog: Logs are sent to the syslog daemon.
13501 · journald: Logs are sent to stderr without timestamp and
13503 log levels compatible with sd-daemon.
13504 You can also specify a full module name here if implement
13506 your own writer:
13507 [log]
13509 writer = stderr
13510 file Specifies the location of file for logging output. Only
13512 used by the file writer:
13513 [log]
13515 file = /var/log/couchdb/couch.log
13516 This path should be readable and writable for user that
13518 runs CouchDB service (couchdb by default).
13519 write_buffer
13521 Specifies the size of the file log write buffer in bytes,
13522 to enable delayed log writes. Only used by the file
13523 writer:
13524 [log]
13526 write_buffer = 0
13527 write_delay
13529 Specifies the wait in milliseconds before committing logs
13530 to disk, to enable delayed log writes. Only used by the
13531 file writer:
13532 [log]
13534 write_delay = 0
13535 level Changed in version 1.3: Added warning level.
13537 Logging level defines how verbose and detailed logging
13540 will be:
13541 [log]
13543 level = info
13544 Available levels:
13546 · debug: Detailed debug logging.
13548 · info: Informative logging. Includes HTTP requests head‐
13550 lines, startup of an external processes etc.
13551 · notice
13553 · warning or warn: Warning messages are alerts about edge
13555 situations that may lead to errors. For instance, com‐
13556 paction daemon alerts about low or insufficient disk
13557 space at this level.
13558 · error or err: Error level includes only things that go
13560 wrong, like crash reports and HTTP error responses (5xx
13561 codes).
13562 · critical or crit
13564 · alert
13566 · emergency or emerg
13568 · none: Disables logging any messages.
13570 include_sasl
13572 Includes SASL information in logs:
13573 [log]
13575 include_sasl = true
13576 syslog_host
13578 Specifies the syslog host to send logs to. Only used by
13579 the syslog writer:
13580 [log]
13582 syslog_host = localhost
13583 syslog_port
13585 Specifies the syslog port to connect to when sending
13586 logs. Only used by the syslog writer:
13587 [log]
13589 syslog_port = 514
13590 syslog_appid
13592 Specifies application name to the syslog writer:
13593 [log]
13595 syslog_appid = couchdb
13596 syslog_facility
13598 Specifies the syslog facility to use with the syslog
13599 writer:
13600 [log]
13602 syslog_facility = local2
13603 Replicator
13605 Replicator Database Configuration
13606 [replicator]
13607 max_jobs
13609 New in version 2.1.
13610 Number of actively running replications. This value rep‐
13613 resents the threshold to trigger the automatic replica‐
13614 tion scheduler. The system will check every interval
13615 milliseconds how many replication jobs are running, and
13616 if there are more than max_jobs active jobs, the sched‐
13617 uler will pause-and-restart up to max_churn jobs in the
13618 scheduler queue. Making this value too high could cause
13619 performance issues, while making it too low could mean
13620 replications jobs might not have enough time to make
13621 progress before getting unscheduled again. This parame‐
13622 ter can be adjusted at runtime and will take effect dur‐
13623 ing next rescheduling cycle:
13624 [replicator]
13626 max_jobs = 500
13627 interval
13629 New in version 2.1.
13630 Scheduling interval in milliseconds. During each
13633 reschedule cycle the scheduler might start or stop up to
13634 max_churn number of jobs:
13635 [replicator]
13637 interval = 60000
13638 max_churn
13640 New in version 2.1.
13641 Maximum number of replication jobs to start and stop dur‐
13644 ing rescheduling. This parameter, along with interval,
13645 defines the rate of job replacement. During startup,
13646 however, a much larger number of jobs could be started
13647 (up to max_jobs) in a short period of time:
13648 [replicator]
13650 max_churn = 20
13651 update_docs
13653 New in version 2.1.
13654 When set to true replicator will update replication docu‐
13657 ment with error and triggered states. This approximates
13658 pre-2.1 replicator behavior:
13659 [replicator]
13661 update_docs = false
13662 worker_batch_size
13664 With lower batch sizes checkpoints are done more fre‐
13665 quently. Lower batch sizes also reduce the total amount
13666 of used RAM memory:
13667 [replicator]
13669 worker_batch_size = 500
13670 worker_processes
13672 More worker processes can give higher network throughput
13673 but can also imply more disk and network IO:
13674 [replicator]
13676 worker_processes = 4
13677 http_connections
13679 Maximum number of HTTP connections per replication:
13680 [replicator]
13682 http_connections = 20
13683 connection_timeout
13685 HTTP connection timeout per replication. This is divided
13686 by three (3) when the replicator makes changes feed
13687 requests. Even for very fast/reliable networks it might
13688 need to be increased if a remote database is too busy:
13689 [replicator]
13691 connection_timeout = 30000
13692 retries_per_request
13694 Changed in version 2.1.1.
13695 If a request fails, the replicator will retry it up to N
13698 times. The default value for N is 5 (before version 2.1.1
13699 it was 10). The requests are retried with a doubling
13700 exponential backoff starting at 0.25 seconds. So by
13701 default requests would be retried in 0.25, 0.5, 1, 2, 4
13702 second intervals. When number of retires is exhausted,
13703 the whole replication job is stopped and will retry again
13704 later:
13705 [replicator]
13707 retries_per_request = 5
13708 socket_options
13710 Some socket options that might boost performance in some
13711 scenarios:
13712 · {nodelay, boolean()}
13714 · {sndbuf, integer()}
13716 · {recbuf, integer()}
13718 · {priority, integer()}
13720 See the inet Erlang module’s man page for the full list
13722 of options:
13723 [replicator]
13725 socket_options = [{keepalive, true}, {nodelay, false}]
13726 checkpoint_interval
13728 New in version 1.6.
13729 Defines replication checkpoint interval in milliseconds.
13732 Replicator will requests from the Source database at the
13733 specified interval:
13734 [replicator]
13736 checkpoint_interval = 5000
13737 Lower intervals may be useful for frequently changing
13739 data, while higher values will lower bandwidth and make
13740 fewer requests for infrequently updated databases.
13741 use_checkpoints
13743 New in version 1.6.
13744 If use_checkpoints is set to true, CouchDB will make
13747 checkpoints during replication and at the completion of
13748 replication. CouchDB can efficiently resume replication
13749 from any of these checkpoints:
13750 [replicator]
13752 use_checkpoints = true
13753 NOTE:
13755 Checkpoints are stored in local documents on both the
13756 source and target databases (which requires write
13757 access).
13758 WARNING:
13760 Disabling checkpoints is not recommended as CouchDB
13761 will scan the Source database’s changes feed from the
13762 beginning.
13763 cert_file
13765 Path to a file containing the user’s certificate:
13766 [replicator]
13768 cert_file = /full/path/to/server_cert.pem
13769 key_file
13771 Path to file containing user’s private PEM encoded key:
13772 [replicator]
13774 key_file = /full/path/to/server_key.pem
13775 password
13777 String containing the user’s password. Only used if the
13778 private key file is password protected:
13779 [replicator]
13781 password = somepassword
13782 verify_ssl_certificates
13784 Set to true to validate peer certificates:
13785 [replicator]
13787 verify_ssl_certificates = false
13788 ssl_trusted_certificates_file
13790 File containing a list of peer trusted certificates (in
13791 the PEM format):
13792 [replicator]
13794 ssl_trusted_certificates_file = /etc/ssl/certs/ca-certificates.crt
13795 ssl_certificate_max_depth
13797 Maximum peer certificate depth (must be set even if cer‐
13798 tificate validation is off):
13799 [replicator]
13801 ssl_certificate_max_depth = 3
13802 auth_plugins
13804 New in version 2.2.
13805 List of replicator client authentication plugins. Plugins
13808 will be tried in order and the first to initialize suc‐
13809 cessfully will be used. By default there are two plugins
13810 available: couch_replicator_auth_session implementing
13811 session (cookie) authentication, and couch_replica‐
13812 tor_auth_noop implementing basic authentication. For
13813 backwards compatibility, the no-op plugin should be used
13814 at the end of the plugin list:
13815 [replicator]
13817 auth_plugins = couch_replicator_auth_session,couch_replicator_auth_noop
13818 NOTE:
13820 In version 2.2, the session plugin is considered
13821 experimental and is not enabled by default.
13822 Query Servers
13824 Query Servers Definition
13825 Changed in version 2.3: Changed configuration method for Query Servers
13826 and Native Query Servers.
13827 CouchDB delegates computation of design documents functions to external
13830 query servers. The external query server is a special OS process which
13831 communicates with CouchDB over standard input/output using a very sim‐
13832 ple line-based protocol with JSON messages.
13833 An external query server may be defined with environment variables fol‐
13835 lowing this pattern:
13836 COUCHDB_QUERY_SERVER_LANGUAGE="PATH ARGS"
13838 Where:
13840 · LANGUAGE: is a programming language which code this query server may
13842 execute. For instance, there are PYTHON, RUBY, CLOJURE and other
13843 query servers in the wild. This value in lowercase is also used for
13844 ddoc field language to determine which query server processes the
13845 functions.
13846 Note, that you may set up multiple query servers for the same pro‐
13848 gramming language, but you have to name them differently (like
13849 PYTHONDEV etc.).
13850 · PATH: is a system path to the executable binary program that runs the
13852 query server.
13853 · ARGS: optionally, you may specify additional command line arguments
13855 for the executable PATH.
13856 The default query server is written in JavaScript, running via Mozilla
13858 SpiderMonkey. It requires no special environment settings to enable,
13859 but is the equivalent of these two variables:
13860 COUCHDB_QUERY_SERVER_JAVASCRIPT="/opt/couchdb/bin/couchjs /opt/couchdb/share/server/main.js"
13862 COUCHDB_QUERY_SERVER_COFFEESCRIPT="/opt/couchdb/bin/couchjs /opt/couchdb/share/server/main-coffee.js"
13863 By default, couchjs limits the max runtime allocation to 64MiB. If you
13865 run into out of memory issue in your ddoc functions, you can adjust the
13866 memory limitation (here, increasing to 512 MiB):
13867 COUCHDB_QUERY_SERVER_JAVASCRIPT="/usr/bin/couchjs -S 536870912 /usr/share/server/main.js"
13869 For more info about the available options, please consult couchjs -h.
13871 SEE ALSO:
13873 The Mango Query Server is a declarative language that requires no
13874 programming, allowing for easier indexing and finding of data in
13875 documents.
13876 The Native Erlang Query Server allows running ddocs written in
13878 Erlang natively, bypassing stdio communication and JSON serializa‐
13879 tion/deserialization round trip overhead.
13880 Query Servers Configuration
13882 [query_server_config]
13883 commit_freq
13885 Specifies the delay in seconds before view index changes
13886 are committed to disk. The default value is 5:
13887 [query_server_config]
13889 commit_freq = 5
13890 os_process_limit
13892 limit Hard limit on the number of OS processes usable by Query
13894 Servers. The default value is 100:
13895 [query_server_config]
13897 os_process_limit = 100
13898 Setting os_process_limit too low can result in starvation
13900 of Query Servers, and manifest in os_process_timeout
13901 errors, while setting it too high can potentially use too
13902 many system resources. Production settings are typically
13903 10-20 times the default value.
13904 os_process_soft_limit
13906 soft limit
13908 Soft limit on the number of OS processes usable by Query
13909 Servers. The default value is 100:
13910 [query_server_config]
13912 os_process_soft_limit = 100
13913 Idle OS processes are closed until the total reaches the
13915 soft limit.
13916 For example, if the hard limit is 200 and the soft limit
13918 is 100, the total number of OS processes will never
13919 exceed 200, and CouchDB will close all idle OS processes
13920 until it reaches 100, at which point it will leave the
13921 rest intact, even if some are idle.
13922 reduce_limit
13924 Controls Reduce overflow error that raises when output of
13925 reduce functions is too big:
13926 [query_server_config]
13928 reduce_limit = true
13929 Normally, you don’t have to disable (by setting false
13931 value) this option since main propose of reduce functions
13932 is to reduce the input.
13933 Native Erlang Query Server
13935 [native_query_servers]
13936 WARNING:
13937 Due to security restrictions, the Erlang query server is dis‐
13938 abled by default.
13939 Unlike the JavaScript query server, the Erlang one does not
13941 runs in a sandbox mode. This means that Erlang code has full
13942 access to your OS, file system and network, which may lead to
13943 security issues. While Erlang functions are faster than
13944 JavaScript ones, you need to be careful about running them,
13945 especially if they were written by someone else.
13946 CouchDB has a native Erlang query server, allowing you to write
13948 your map/reduce functions in Erlang.
13949 First, you’ll need to edit your local.ini to include a
13951 [native_query_servers] section:
13952 [native_query_servers]
13954 enable_erlang_query_server = true
13955 To see these changes you will also need to restart the server.
13957 Let’s try an example of map/reduce functions which count the
13959 total documents at each number of revisions (there are x many
13960 documents at version “1”, and y documents at “2”… etc). Add a
13961 few documents to the database, then enter the following func‐
13962 tions as a view:
13963 %% Map Function
13965 fun({Doc}) ->
13966 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
13967 V = proplists:get_value(<<"_id">>, Doc, null),
13968 Emit(<<K>>, V)
13969 end.
13970 %% Reduce Function
13972 fun(Keys, Values, ReReduce) -> length(Values) end.
13973 If all has gone well, after running the view you should see a
13975 list of the total number of documents at each revision number.
13976 Additional examples are on the users@couchdb.apache.org mailing
13978 list.
13979 Search
13981 CouchDB’s search subsystem can be configured via the dreyfus configura‐
13982 tion section.
13983 [dreyfus]
13985 name The name and location of the Clouseau Java service
13987 required to enable Search functionality. Defaults to
13988 clouseau@127.0.0.1.
13989 retry_limit
13991 CouchDB will try to reconnect to Clouseau using a bounded
13992 exponential backoff with the following number of itera‐
13993 tions. Defaults to 5.
13994 limit The number of results returned from a global search query
13996 if no limit is specified. Defaults to 25.
13997 limit_partitions
13999 The number of results returned from a search on a parti‐
14000 tion of a database if no limit is specified. Defaults to
14001 2000.
14002 max_limit
14004 The maximum number of results that can be returned from a
14005 global search query (or any search query on a database
14006 without user-defined partitions). Attempts to set
14007 ?limit=N higher than this value will be rejected.
14008 Defaults to 200.
14009 max_limit_partitions
14011 The maximum number of results that can be returned when
14012 searching a partition of a database. Attempts to set
14013 ?limit=N higher than this value will be rejected. If this
14014 config setting is not defined, CouchDB will use the value
14015 of max_limit instead. If neither is defined, the default
14016 is 2000.
14017 Mango
14019 Mango is the Query Engine that services the _find, endpoint.
14020 [mango]
14022 index_all_disabled
14024 Set to true to disable the “index all fields” text index.
14025 This can lead to out of memory issues when there are doc‐
14026 uments with nested array fields. Defaults to false.
14027 [mango] index_all_disabled = false
14028 default_limit
14030 Sets the default number of results that will be returned
14031 in a _find response. Individual requests can override
14032 this by setting limit directly in the query parameters.
14033 Defaults to 25.
14034 [mango] default_limit = 25
14035 index_scan_warning_threshold
14037 index scan warning
14039 This sets the ratio between documents scanned and
14040 results matched that will generate a warning in the
14041 _find response. For example, if a query requires read‐
14042 ing 100 documents to return 10 rows, a warning will be
14043 generated if this value is 10.
14044 Defaults to 10. Setting the value to 0 disables the
14046 warning.
14047 [mango] index_scan_warning_threshold = 10
14048 Miscellaneous Parameters
14050 Configuration of Attachment Storage
14051 [attachments]
14052 compression_level
14054 Defines zlib compression level for the attachments from 1
14055 (lowest, fastest) to 9 (highest, slowest). A value of 0
14056 disables compression:
14057 [attachments]
14059 compression_level = 8
14060 compressible_types
14062 Since compression is ineffective for some types of files,
14063 it is possible to let CouchDB compress only some types of
14064 attachments, specified by their MIME type:
14065 [attachments]
14067 compressible_types = text/*, application/javascript, application/json, application/xml
14068 Statistic Calculation
14070 [stats]
14071 interval
14073 Interval between gathering statistics in seconds:
14074 [stats]
14076 interval = 10
14077 UUIDs Configuration
14079 [uuids]
14080 algorithm
14082 Changed in version 1.3: Added utc_id algorithm.
14083 CouchDB provides various algorithms to generate the UUID
14086 values that are used for document _id’s by default:
14087 [uuids]
14089 algorithm = sequential
14090 Available algorithms:
14092 · random: 128 bits of random awesome. All awesome, all
14094 the time:
14095 {
14097 "uuids": [
14098 "5fcbbf2cb171b1d5c3bc6df3d4affb32",
14099 "9115e0942372a87a977f1caf30b2ac29",
14100 "3840b51b0b81b46cab99384d5cd106e3",
14101 "b848dbdeb422164babf2705ac18173e1",
14102 "b7a8566af7e0fc02404bb676b47c3bf7",
14103 "a006879afdcae324d70e925c420c860d",
14104 "5f7716ee487cc4083545d4ca02cd45d4",
14105 "35fdd1c8346c22ccc43cc45cd632e6d6",
14106 "97bbdb4a1c7166682dc026e1ac97a64c",
14107 "eb242b506a6ae330bda6969bb2677079"
14108 ]
14109 }
14110 · sequential: Monotonically increasing ids with random
14112 increments. The first 26 hex characters are random,
14113 the last 6 increment in random amounts until an over‐
14114 flow occurs. On overflow, the random prefix is regener‐
14115 ated and the process starts over.
14116 {
14118 "uuids": [
14119 "4e17c12963f4bee0e6ec90da54804894",
14120 "4e17c12963f4bee0e6ec90da5480512f",
14121 "4e17c12963f4bee0e6ec90da54805c25",
14122 "4e17c12963f4bee0e6ec90da54806ba1",
14123 "4e17c12963f4bee0e6ec90da548072b3",
14124 "4e17c12963f4bee0e6ec90da54807609",
14125 "4e17c12963f4bee0e6ec90da54807718",
14126 "4e17c12963f4bee0e6ec90da54807754",
14127 "4e17c12963f4bee0e6ec90da54807e5d",
14128 "4e17c12963f4bee0e6ec90da54808d28"
14129 ]
14130 }
14131 · utc_random: The time since Jan 1, 1970 UTC, in
14133 microseconds. The first 14 characters are the time in
14134 hex. The last 18 are random.
14135 {
14137 "uuids": [
14138 "04dd32b3af699659b6db9486a9c58c62",
14139 "04dd32b3af69bb1c2ac7ebfee0a50d88",
14140 "04dd32b3af69d8591b99a8e86a76e0fb",
14141 "04dd32b3af69f4a18a76efd89867f4f4",
14142 "04dd32b3af6a1f7925001274bbfde952",
14143 "04dd32b3af6a3fe8ea9b120ed906a57f",
14144 "04dd32b3af6a5b5c518809d3d4b76654",
14145 "04dd32b3af6a78f6ab32f1e928593c73",
14146 "04dd32b3af6a99916c665d6bbf857475",
14147 "04dd32b3af6ab558dd3f2c0afacb7d66"
14148 ]
14149 }
14150 · utc_id: The time since Jan 1, 1970 UTC, in microsec‐
14152 onds, plus the utc_id_suffix string. The first 14 char‐
14153 acters are the time in hex. The uuids/utc_id_suffix
14154 string value is appended to these.
14155 {
14157 "uuids": [
14158 "04dd32bd5eabcc@mycouch",
14159 "04dd32bd5eabee@mycouch",
14160 "04dd32bd5eac05@mycouch",
14161 "04dd32bd5eac28@mycouch",
14162 "04dd32bd5eac43@mycouch",
14163 "04dd32bd5eac58@mycouch",
14164 "04dd32bd5eac6e@mycouch",
14165 "04dd32bd5eac84@mycouch",
14166 "04dd32bd5eac98@mycouch",
14167 "04dd32bd5eacad@mycouch"
14168 ]
14169 }
14170 NOTE:
14172 Impact of UUID choices: the choice of UUID has a sig‐
14173 nificant impact on the layout of the B-tree, prior to
14174 compaction.
14175 For example, using a sequential UUID algorithm while
14177 uploading a large batch of documents will avoid the
14178 need to rewrite many intermediate B-tree nodes. A ran‐
14179 dom UUID algorithm may require rewriting intermediate
14180 nodes on a regular basis, resulting in significantly
14181 decreased throughput and wasted disk space space due
14182 to the append-only B-tree design.
14183 It is generally recommended to set your own UUIDs, or
14185 use the sequential algorithm unless you have a spe‐
14186 cific need and take into account the likely need for
14187 compaction to re-balance the B-tree and reclaim wasted
14188 space.
14189 utc_id_suffix
14191 New in version 1.3.
14192 The utc_id_suffix value will be appended to UUIDs gener‐
14195 ated by the utc_id algorithm. Replicating instances
14196 should have unique utc_id_suffix values to ensure unique‐
14197 ness of utc_id ids.
14198 [uuid]
14200 utc_id_suffix = my-awesome-suffix
14201 max_count
14203 New in version 1.5.1.
14204 No more than this number of UUIDs will be sent in a sin‐
14207 gle request. If more UUIDs are requested, an HTTP error
14208 response will be thrown.
14209 [uuid]
14211 max_count = 1000
14212 Vendor information
14214 [vendor]
14215 New in version 1.3.
14216 CouchDB distributors have the option of customizing CouchDB’s
14219 welcome message. This is returned when requesting GET /.
14220 [vendor]
14222 name = The Apache Software Foundation
14223 version = 1.5.0
14224 Content-Security-Policy
14226 [csp] Experimental support of CSP Headers for /_utils (Fauxton).
14227 enable Enable the sending of the Header Content-Security-Policy:
14229 [csp]
14231 enable = true
14232 header_value
14234 You can change the default value for the Header which is
14235 sent:
14236 [csp]
14238 header_value = default-src 'self'; img-src *; font-src *;
14239 Configuration of Database Purge
14241 [purge]
14242 max_document_id_number
14244 New in version 3.0.
14245 Sets the maximum number of documents allowed in a single
14248 purge request:
14249 [purge]
14251 max_document_id_number = 100
14252 max_revisions_number
14254 New in version 3.0.
14255 Sets the maximum number of accumulated revisions allowed
14258 in a single purge request:
14259 [purge]
14261 max_revisions_number = 1000
14262 index_lag_warn_seconds
14264 New in version 3.0.
14265 Sets the allowed duration when index is not updated for
14268 local purge checkpoint document. Default is 24 hours:
14269 [purge]
14271 index_lag_warn_seconds = 86400
14272 Resharding
14274 Resharding Configuration
14275 [resharding]
14276 max_jobs
14278 Maximum number of resharding jobs per cluster node. This
14279 includes completed, failed, and running jobs. If the job
14280 appears in the _reshard/jobs HTTP API results it will be
14281 counted towards the limit. When more than max_jobs jobs
14282 have been created, subsequent requests will start to fail
14283 with the max_jobs_exceeded error:
14284 [reshard]
14286 max_jobs = 48
14287 max_history
14289 Each resharding job maintains a timestamped event log.
14290 This setting limits the maximum size of that log:
14291 [reshard]
14293 max_history = 20
14294 max_retries
14296 How many times to retry shard splitting steps if they
14297 fail. For example, if indexing or topping off fails, it
14298 will be retried up to this many times before the whole
14299 resharding job fails:
14300 [reshard]
14302 max_retries = 1
14303 retry_interval_sec
14305 How long to wait between subsequent retries:
14306 [reshard]
14308 retry_interval_sec = 10
14309 delete_source
14311 Indicates if the source shard should be deleted after
14312 resharding has finished. By default, it is true as that
14313 would recover the space utilized by the shard. When
14314 debugging or when extra safety is required, this can be
14315 switched to false:
14316 [reshard]
14318 delete_source = true
14319 update_shard_map_timeout_sec
14321 How many seconds to wait for the shard map update opera‐
14322 tion to complete. If there is a large number of shard db
14323 changes waiting to finish replicating, it might be bene‐
14324 ficial to increase this timeout:
14325 [reshard]
14327 update_shard_map_timeout_sec = 60
14328 source_close_timeout_sec
14330 How many seconds to wait for the source shard to close.
14331 “Close” in this context means that client requests which
14332 keep the database open have all finished:
14333 [reshard]
14335 source_close_timeout_sec = 600
14336 require_node_param
14338 Require users to specify a node parameter when creating
14339 resharding jobs. This can be used as a safety check to
14340 avoid inadvertently starting too many resharding jobs by
14341 accident:
14342 [reshard]
14344 require_node_param = false
14345 require_range_param
14347 Require users to specify a range parameter when creating
14348 resharding jobs. This can be used as a safety check to
14349 avoid inadvertently starting too many resharding jobs by
14350 accident:
14351 [reshard]
14353 require_range_param = false
14354
14356 As of CouchDB 2.0.0, CouchDB can be run in two different modes of oper‐
14357 ation:
14358 · Standalone
14360 · Cluster
14362 This section details the theory behind CouchDB clusters, and provides
14364 specific operational instructions on node, database and shard manage‐
14365 ment.
14366 Theory
14368 Before we move on, we need some theory.
14369 As you see in etc/default.ini there is a section called [cluster]
14371 [cluster]
14373 q=2
14374 n=3
14375 · q - The number of shards.
14377 · n - The number of copies there is of every document. Replicas.
14379 When creating a database you can send your own values with request and
14381 thereby override the defaults in default.ini.
14382 In clustered operation, a quorum must be reached before CouchDB returns
14384 a 200 for a fetch, or 201 for a write operation. A quorum is defined as
14385 one plus half the number of “relevant copies”. “Relevant copies” is
14386 defined slightly differently for read and write operations.
14387 For read operations, the number of relevant copies is the number of
14389 currently-accessible shards holding the requested data, meaning that in
14390 the case of a failure or network partition, the number of relevant
14391 copies may be lower than the number of replicas in the cluster. The
14392 number of read copies can be set with the r parameter.
14393 For write operations the number of relevant copies is always n, the
14395 number of replicas in the cluster. For write operations, the number of
14396 copies can be set using the w parameter. If fewer than this number of
14397 nodes is available, a 202 will be returned.
14398 We will focus on the shards and replicas for now.
14400 A shard is a part of a database. It can be replicated multiple times.
14402 The more copies of a shard, the more you can scale out. If you have 4
14403 replicas, that means that all 4 copies of this specific shard will live
14404 on at most 4 nodes. With one replica you can have only one node, just
14405 as with CouchDB 1.x. No node can have more than one copy of each shard
14406 replica. The default for CouchDB since 3.0.0 is q=2 and n=3, meaning
14407 each database (and secondary index) is split into 2 shards, with 3
14408 replicas per shard, for a total of 6 shard replica files. For a CouchDB
14409 cluster only hosting a single database with these default values, a
14410 maximum of 6 nodes can be used to scale horizontally.
14411 Replicas add failure resistance, as some nodes can be offline without
14413 everything crashing down.
14414 · n=1 All nodes must be up.
14416 · n=2 Any 1 node can be down.
14418 · n=3 Any 2 nodes can be down.
14420 · etc
14422 Computers go down and sysadmins pull out network cables in a furious
14424 rage from time to time, so using n<2 is asking for downtime. Having too
14425 high a value of n adds servers and complexity without any real benefit.
14426 The sweet spot is at n=3.
14427 Say that we have a database with 3 replicas and 4 shards. That would
14429 give us a maximum of 12 nodes: 4*3=12.
14430 We can lose any 2 nodes and still read and write all documents.
14432 What happens if we lose more nodes? It depends on how lucky we are. As
14434 long as there is at least one copy of every shard online, we can read
14435 and write all documents.
14436 So, if we are very lucky then we can lose 8 nodes at maximum.
14438 Node Management
14440 Adding a node
14441 Go to http://server1:5984/_membership to see the name of the node and
14442 all the nodes it is connected to and knows about.
14443 curl -X GET "http://xxx.xxx.xxx.xxx:5984/_membership" --user admin-user
14445 {
14447 "all_nodes":[
14448 "node1@xxx.xxx.xxx.xxx"],
14449 "cluster_nodes":[
14450 "node1@xxx.xxx.xxx.xxx"]
14451 }
14452 · all_nodes are all the nodes thats this node knows about.
14454 · cluster_nodes are the nodes that are connected to this node.
14456 To add a node simply do:
14458 curl -X PUT "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy" -d {}
14460 Now look at http://server1:5984/_membership again.
14462 {
14464 "all_nodes":[
14465 "node1@xxx.xxx.xxx.xxx",
14466 "node2@yyy.yyy.yyy.yyy"
14467 ],
14468 "cluster_nodes":[
14469 "node1@xxx.xxx.xxx.xxx",
14470 "node2@yyy.yyy.yyy.yyy"
14471 ]
14472 }
14473 And you have a 2 node cluster :)
14475 http://yyy.yyy.yyy.yyy:5984/_membership will show the same thing, so
14477 you only have to add a node once.
14478 Removing a node
14480 Before you remove a node, make sure that you have moved all shards away
14481 from that node.
14482 To remove node2 from server yyy.yyy.yyy.yyy, you need to first know the
14484 revision of the document that signifies that node’s existence:
14485 curl "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy"
14487 {"_id":"node2@yyy.yyy.yyy.yyy","_rev":"1-967a00dff5e02add41820138abb3284d"}
14488 With that _rev, you can now proceed to delete the node document:
14490 curl -X DELETE "http://xxx.xxx.xxx.xxx/_node/_local/_nodes/node2@yyy.yyy.yyy.yyy?rev=1-967a00dff5e02add41820138abb3284d"
14492 Database Management
14494 Creating a database
14495 This will create a database with 3 replicas and 8 shards.
14496 curl -X PUT "http://xxx.xxx.xxx.xxx:5984/database-name?n=3&q=8" --user admin-user
14498 The database is in data/shards. Look around on all the nodes and you
14500 will find all the parts.
14501 If you do not specify n and q the default will be used. The default is
14503 3 replicas and 8 shards.
14504 Deleting a database
14506 curl -X DELETE "http://xxx.xxx.xxx.xxx:5984/database-name --user admin-user
14507 Placing a database on specific nodes
14509 In BigCouch, the predecessor to CouchDB 2.0’s clustering functionality,
14510 there was the concept of zones. CouchDB 2.0 carries this forward with
14511 cluster placement rules.
14512 WARNING:
14514 Use of the placement argument will override the standard logic for
14515 shard replica cardinality (specified by [cluster] n.)
14516 First, each node must be labeled with a zone attribute. This defines
14518 which zone each node is in. You do this by editing the node’s document
14519 in the system _nodes database, which is accessed node-local via the GET
14520 /_node/_local/_nodes/{node-name} endpoint.
14521 Add a key value pair of the form:
14523 "zone": "metro-dc-a"
14525 Do this for all of the nodes in your cluster.
14527 In your config file (local.ini or default.ini) on each node, define a
14529 consistent cluster-wide setting like:
14530 [cluster]
14532 placement = metro-dc-a:2,metro-dc-b:1
14533 In this example, it will ensure that two replicas for a shard will be
14535 hosted on nodes with the zone attribute set to metro-dc-a and one
14536 replica will be hosted on a new with the zone attribute set to
14537 metro-dc-b.
14538 Note that you can also use this system to ensure certain nodes in the
14540 cluster do not host any replicas for newly created databases, by giving
14541 them a zone attribute that does not appear in the [cluster] placement
14542 string.
14543 Shard Management
14545 Introduction
14546 This document discusses how sharding works in CouchDB along with how to
14547 safely add, move, remove, and create placement rules for shards and
14548 shard replicas.
14549 A shard is a horizontal partition of data in a database. Partitioning
14551 data into shards and distributing copies of each shard (called “shard
14552 replicas” or just “replicas”) to different nodes in a cluster gives the
14553 data greater durability against node loss. CouchDB clusters automati‐
14554 cally shard databases and distribute the subsets of documents that com‐
14555 pose each shard among nodes. Modifying cluster membership and sharding
14556 behavior must be done manually.
14557 Shards and Replicas
14559 How many shards and replicas each database has can be set at the global
14560 level, or on a per-database basis. The relevant parameters are q and n.
14561 q is the number of database shards to maintain. n is the number of
14563 copies of each document to distribute. The default value for n is 3,
14564 and for q is 2. With q=2, the database is split into 2 shards. With
14565 n=3, the cluster distributes three replicas of each shard. Altogether,
14566 that’s 6 shard replicas for a single database.
14567 In a 3-node cluster with q=8, each node would receive 8 shards. In a
14569 4-node cluster, each node would receive 6 shards. We recommend in the
14570 general case that the number of nodes in your cluster should be a mul‐
14571 tiple of n, so that shards are distributed evenly.
14572 CouchDB nodes have a etc/default.ini file with a section named cluster
14574 which looks like this:
14575 [cluster]
14577 q=2
14578 n=3
14579 These settings specify the default sharding parameters for newly cre‐
14581 ated databases. These can be overridden in the etc/local.ini file by
14582 copying the text above, and replacing the values with your new
14583 defaults. The values can also be set on a per-database basis by speci‐
14584 fying the q and n query parameters when the database is created. For
14585 example:
14586 $ curl -X PUT "$COUCH_URL:5984/database-name?q=4&n=2"
14588 This creates a database that is split into 4 shards and 2 replicas,
14590 yielding 8 shard replicas distributed throughout the cluster.
14591 Quorum
14593 Depending on the size of the cluster, the number of shards per data‐
14594 base, and the number of shard replicas, not every node may have access
14595 to every shard, but every node knows where all the replicas of each
14596 shard can be found through CouchDB’s internal shard map.
14597 Each request that comes in to a CouchDB cluster is handled by any one
14599 random coordinating node. This coordinating node proxies the request to
14600 the other nodes that have the relevant data, which may or may not
14601 include itself. The coordinating node sends a response to the client
14602 once a quorum of database nodes have responded; 2, by default. The
14603 default required size of a quorum is equal to r=w=((n+1)/2) where r
14604 refers to the size of a read quorum, w refers to the size of a write
14605 quorum, and n refers to the number of replicas of each shard. In a
14606 default cluster where n is 3, ((n+1)/2) would be 2.
14607 NOTE:
14609 Each node in a cluster can be a coordinating node for any one
14610 request. There are no special roles for nodes inside the cluster.
14611 The size of the required quorum can be configured at request time by
14613 setting the r parameter for document and view reads, and the w parame‐
14614 ter for document writes. For example, here is a request that directs
14615 the coordinating node to send a response once at least two nodes have
14616 responded:
14617 $ curl "$COUCH_URL:5984/{db}/{doc}?r=2"
14619 Here is a similar example for writing a document:
14621 $ curl -X PUT "$COUCH_URL:5984/{db}/{doc}?w=2" -d '{...}'
14623 Setting r or w to be equal to n (the number of replicas) means you will
14625 only receive a response once all nodes with relevant shards have
14626 responded or timed out, and as such this approach does not guarantee
14627 ACIDic consistency. Setting r or w to 1 means you will receive a
14628 response after only one relevant node has responded.
14629 Examining database shards
14631 There are a few API endpoints that help you understand how a database
14632 is sharded. Let’s start by making a new database on a cluster, and
14633 putting a couple of documents into it:
14634 $ curl -X PUT $COUCH_URL:5984/mydb
14636 {"ok":true}
14637 $ curl -X PUT $COUCH_URL:5984/mydb/joan -d '{"loves":"cats"}'
14638 {"ok":true,"id":"joan","rev":"1-cc240d66a894a7ee7ad3160e69f9051f"}
14639 $ curl -X PUT $COUCH_URL:5984/mydb/robert -d '{"loves":"dogs"}'
14640 {"ok":true,"id":"robert","rev":"1-4032b428c7574a85bc04f1f271be446e"}
14641 First, the top level api/db endpoint will tell you what the sharding
14643 parameters are for your database:
14644 $ curl -s $COUCH_URL:5984/db | jq .
14646 {
14647 "db_name": "mydb",
14648 ...
14649 "cluster": {
14650 "q": 8,
14651 "n": 3,
14652 "w": 2,
14653 "r": 2
14654 },
14655 ...
14656 }
14657 So we know this database was created with 8 shards (q=8), and each
14659 shard has 3 replicas (n=3) for a total of 24 shard replicas across the
14660 nodes in the cluster.
14661 Now, let’s see how those shard replicas are placed on the cluster with
14663 the api/db/shards endpoint:
14664 $ curl -s $COUCH_URL:5984/mydb/_shards | jq .
14666 {
14667 "shards": {
14668 "00000000-1fffffff": [
14669 "node1@127.0.0.1",
14670 "node2@127.0.0.1",
14671 "node4@127.0.0.1"
14672 ],
14673 "20000000-3fffffff": [
14674 "node1@127.0.0.1",
14675 "node2@127.0.0.1",
14676 "node3@127.0.0.1"
14677 ],
14678 "40000000-5fffffff": [
14679 "node2@127.0.0.1",
14680 "node3@127.0.0.1",
14681 "node4@127.0.0.1"
14682 ],
14683 "60000000-7fffffff": [
14684 "node1@127.0.0.1",
14685 "node3@127.0.0.1",
14686 "node4@127.0.0.1"
14687 ],
14688 "80000000-9fffffff": [
14689 "node1@127.0.0.1",
14690 "node2@127.0.0.1",
14691 "node4@127.0.0.1"
14692 ],
14693 "a0000000-bfffffff": [
14694 "node1@127.0.0.1",
14695 "node2@127.0.0.1",
14696 "node3@127.0.0.1"
14697 ],
14698 "c0000000-dfffffff": [
14699 "node2@127.0.0.1",
14700 "node3@127.0.0.1",
14701 "node4@127.0.0.1"
14702 ],
14703 "e0000000-ffffffff": [
14704 "node1@127.0.0.1",
14705 "node3@127.0.0.1",
14706 "node4@127.0.0.1"
14707 ]
14708 }
14709 }
14710 Now we see that there are actually 4 nodes in this cluster, and CouchDB
14712 has spread those 24 shard replicas evenly across all 4 nodes.
14713 We can also see exactly which shard contains a given document with the
14715 api/db/shards/doc endpoint:
14716 $ curl -s $COUCH_URL:5984/mydb/_shards/joan | jq .
14718 {
14719 "range": "e0000000-ffffffff",
14720 "nodes": [
14721 "node1@127.0.0.1",
14722 "node3@127.0.0.1",
14723 "node4@127.0.0.1"
14724 ]
14725 }
14726 $ curl -s $COUCH_URL:5984/mydb/_shards/robert | jq .
14727 {
14728 "range": "60000000-7fffffff",
14729 "nodes": [
14730 "node1@127.0.0.1",
14731 "node3@127.0.0.1",
14732 "node4@127.0.0.1"
14733 ]
14734 }
14735 CouchDB shows us the specific shard into which each of the two sample
14737 documents is mapped.
14738 Moving a shard
14740 When moving shards or performing other shard manipulations on the clus‐
14741 ter, it is advisable to stop all resharding jobs on the cluster. See
14742 Stopping Resharding Jobs for more details.
14743 This section describes how to manually place and replace shards. These
14745 activities are critical steps when you determine your cluster is too
14746 big or too small, and want to resize it successfully, or you have
14747 noticed from server metrics that database/shard layout is non-optimal
14748 and you have some “hot spots” that need resolving.
14749 Consider a three-node cluster with q=8 and n=3. Each database has 24
14751 shards, distributed across the three nodes. If you add a fourth node to
14752 the cluster, CouchDB will not redistribute existing database shards to
14753 it. This leads to unbalanced load, as the new node will only host
14754 shards for databases created after it joined the cluster. To balance
14755 the distribution of shards from existing databases, they must be moved
14756 manually.
14757 Moving shards between nodes in a cluster involves the following steps:
14759 0. Ensure the target node has joined the cluster.
14761 1. Copy the shard(s) and any secondary index shard(s) onto the target
14763 node.
14764 2. Set the target node to maintenance mode.
14766 3. Update cluster metadata to reflect the new target shard(s).
14768 4. Monitor internal replication to ensure up-to-date shard(s).
14770 5. Clear the target node’s maintenance mode.
14772 6. Update cluster metadata again to remove the source shard(s)
14774 7. Remove the shard file(s) and secondary index file(s) from the
14776 source node.
14777 Copying shard files
14779 NOTE:
14780 Technically, copying database and secondary index shards is
14781 optional. If you proceed to the next step without performing this
14782 data copy, CouchDB will use internal replication to populate the
14783 newly added shard replicas. However, copying files is faster than
14784 internal replication, especially on a busy cluster, which is why we
14785 recommend performing this manual data copy first.
14786 Shard files live in the data/shards directory of your CouchDB install.
14788 Within those subdirectories are the shard files themselves. For
14789 instance, for a q=8 database called abc, here is its database shard
14790 files:
14791 data/shards/00000000-1fffffff/abc.1529362187.couch
14793 data/shards/20000000-3fffffff/abc.1529362187.couch
14794 data/shards/40000000-5fffffff/abc.1529362187.couch
14795 data/shards/60000000-7fffffff/abc.1529362187.couch
14796 data/shards/80000000-9fffffff/abc.1529362187.couch
14797 data/shards/a0000000-bfffffff/abc.1529362187.couch
14798 data/shards/c0000000-dfffffff/abc.1529362187.couch
14799 data/shards/e0000000-ffffffff/abc.1529362187.couch
14800 Secondary indexes (including JavaScript views, Erlang views and Mango
14802 indexes) are also sharded, and their shards should be moved to save the
14803 new node the effort of rebuilding the view. View shards live in
14804 data/.shards. For example:
14805 data/.shards
14807 data/.shards/e0000000-ffffffff/_replicator.1518451591_design
14808 data/.shards/e0000000-ffffffff/_replicator.1518451591_design/mrview
14809 data/.shards/e0000000-ffffffff/_replicator.1518451591_design/mrview/3e823c2a4383ac0c18d4e574135a5b08.view
14810 data/.shards/c0000000-dfffffff
14811 data/.shards/c0000000-dfffffff/_replicator.1518451591_design
14812 data/.shards/c0000000-dfffffff/_replicator.1518451591_design/mrview
14813 data/.shards/c0000000-dfffffff/_replicator.1518451591_design/mrview/3e823c2a4383ac0c18d4e574135a5b08.view
14814 ...
14815 Since they are files, you can use cp, rsync, scp or other file-copying
14817 command to copy them from one node to another. For example:
14818 # one one machine
14820 $ mkdir -p data/.shards/{range}
14821 $ mkdir -p data/shards/{range}
14822 # on the other
14823 $ scp {couch-dir}/data/.shards/{range}/{database}.{datecode}* \
14824 {node}:{couch-dir}/data/.shards/{range}/
14825 $ scp {couch-dir}/data/shards/{range}/{database}.{datecode}.couch \
14826 {node}:{couch-dir}/data/shards/{range}/
14827 NOTE:
14829 Remember to move view files before database files! If a view index
14830 is ahead of its database, the database will rebuild it from scratch.
14831 Set the target node to true maintenance mode
14833 Before telling CouchDB about these new shards on the node, the node
14834 must be put into maintenance mode. Maintenance mode instructs CouchDB
14835 to return a 404 Not Found response on the /_up endpoint, and ensures it
14836 does not participate in normal interactive clustered requests for its
14837 shards. A properly configured load balancer that uses GET /_up to check
14838 the health of nodes will detect this 404 and remove the node from cir‐
14839 culation, preventing requests from being sent to that node. For exam‐
14840 ple, to configure HAProxy to use the /_up endpoint, use:
14841 http-check disable-on-404
14843 option httpchk GET /_up
14844 If you do not set maintenance mode, or the load balancer ignores this
14846 maintenance mode status, after the next step is performed the cluster
14847 may return incorrect responses when consulting the node in question.
14848 You don’t want this! In the next steps, we will ensure that this shard
14849 is up-to-date before allowing it to participate in end-user requests.
14850 To enable maintenance mode:
14852 $ curl -X PUT -H "Content-type: application/json" \
14854 $COUCH_URL:5984/_node/{node-name}/_config/couchdb/maintenance_mode \
14855 -d "\"true\""
14856 Then, verify that the node is in maintenance mode by performing a GET
14858 /_up on that node’s individual endpoint:
14859 $ curl -v $COUCH_URL/_up
14861 …
14862 < HTTP/1.1 404 Object Not Found
14863 …
14864 {"status":"maintenance_mode"}
14865 Finally, check that your load balancer has removed the node from the
14867 pool of available backend nodes.
14868 Updating cluster metadata to reflect the new target shard(s)
14870 Now we need to tell CouchDB that the target node (which must already be
14871 joined to the cluster) should be hosting shard replicas for a given
14872 database.
14873 To update the cluster metadata, use the special /_dbs database, which
14875 is an internal CouchDB database that maps databases to shards and
14876 nodes. This database is automatically replicated between nodes. It is
14877 accessible only through the special /_node/_local/_dbs endpoint.
14878 First, retrieve the database’s current metadata:
14880 $ curl http://localhost/_node/_local/_dbs/{name}
14882 {
14883 "_id": "{name}",
14884 "_rev": "1-e13fb7e79af3b3107ed62925058bfa3a",
14885 "shard_suffix": [46, 49, 53, 51, 48, 50, 51, 50, 53, 50, 54],
14886 "changelog": [
14887 ["add", "00000000-1fffffff", "node1@xxx.xxx.xxx.xxx"],
14888 ["add", "00000000-1fffffff", "node2@xxx.xxx.xxx.xxx"],
14889 ["add", "00000000-1fffffff", "node3@xxx.xxx.xxx.xxx"],
14890 …
14891 ],
14892 "by_node": {
14893 "node1@xxx.xxx.xxx.xxx": [
14894 "00000000-1fffffff",
14895 …
14896 ],
14897 …
14898 },
14899 "by_range": {
14900 "00000000-1fffffff": [
14901 "node1@xxx.xxx.xxx.xxx",
14902 "node2@xxx.xxx.xxx.xxx",
14903 "node3@xxx.xxx.xxx.xxx"
14904 ],
14905 …
14906 }
14907 }
14908 Here is a brief anatomy of that document:
14910 · _id: The name of the database.
14912 · _rev: The current revision of the metadata.
14914 · shard_suffix: A timestamp of the database’s creation, marked as sec‐
14916 onds after the Unix epoch mapped to the codepoints for ASCII numer‐
14917 als.
14918 · changelog: History of the database’s shards.
14920 · by_node: List of shards on each node.
14922 · by_range: On which nodes each shard is.
14924 To reflect the shard move in the metadata, there are three steps:
14926 1. Add appropriate changelog entries.
14928 2. Update the by_node entries.
14930 3. Update the by_range entries.
14932 WARNING:
14934 Be very careful! Mistakes during this process can irreparably cor‐
14935 rupt the cluster!
14936 As of this writing, this process must be done manually.
14938 To add a shard to a node, add entries like this to the database meta‐
14940 data’s changelog attribute:
14941 ["add", "{range}", "{node-name}"]
14943 The {range} is the specific shard range for the shard. The {node-name}
14945 should match the name and address of the node as displayed in GET
14946 /_membership on the cluster.
14947 NOTE:
14949 When removing a shard from a node, specify remove instead of add.
14950 Once you have figured out the new changelog entries, you will need to
14952 update the by_node and by_range to reflect who is storing what shards.
14953 The data in the changelog entries and these attributes must match. If
14954 they do not, the database may become corrupted.
14955 Continuing our example, here is an updated version of the metadata
14957 above that adds shards to an additional node called node4:
14958 {
14960 "_id": "{name}",
14961 "_rev": "1-e13fb7e79af3b3107ed62925058bfa3a",
14962 "shard_suffix": [46, 49, 53, 51, 48, 50, 51, 50, 53, 50, 54],
14963 "changelog": [
14964 ["add", "00000000-1fffffff", "node1@xxx.xxx.xxx.xxx"],
14965 ["add", "00000000-1fffffff", "node2@xxx.xxx.xxx.xxx"],
14966 ["add", "00000000-1fffffff", "node3@xxx.xxx.xxx.xxx"],
14967 ...
14968 ["add", "00000000-1fffffff", "node4@xxx.xxx.xxx.xxx"]
14969 ],
14970 "by_node": {
14971 "node1@xxx.xxx.xxx.xxx": [
14972 "00000000-1fffffff",
14973 ...
14974 ],
14975 ...
14976 "node4@xxx.xxx.xxx.xxx": [
14977 "00000000-1fffffff"
14978 ]
14979 },
14980 "by_range": {
14981 "00000000-1fffffff": [
14982 "node1@xxx.xxx.xxx.xxx",
14983 "node2@xxx.xxx.xxx.xxx",
14984 "node3@xxx.xxx.xxx.xxx",
14985 "node4@xxx.xxx.xxx.xxx"
14986 ],
14987 ...
14988 }
14989 }
14990 Now you can PUT this new metadata:
14992 $ curl -X PUT http://localhost/_node/_local/_dbs/{name} -d '{...}'
14994 Forcing synchronization of the shard(s)
14996 New in version 2.4.0.
14997 Whether you pre-copied shards to your new node or not, you can force
15000 CouchDB to synchronize all replicas of all shards in a database with
15001 the api/db/sync_shards endpoint:
15002 $ curl -X POST $COUCH_URL:5984/{db}/_sync_shards
15004 {"ok":true}
15005 This starts the synchronization process. Note that this will put addi‐
15007 tional load onto your cluster, which may affect performance.
15008 It is also possible to force synchronization on a per-shard basis by
15010 writing to a document that is stored within that shard.
15011 NOTE:
15013 Admins may want to bump their [mem3] sync_concurrency value to a
15014 larger figure for the duration of the shards sync.
15015 Monitor internal replication to ensure up-to-date shard(s)
15017 After you complete the previous step, CouchDB will have started syn‐
15018 chronizing the shards. You can observe this happening by monitoring the
15019 /_node/{node-name}/_system endpoint, which includes the internal_repli‐
15020 cation_jobs metric.
15021 Once this metric has returned to the baseline from before you started
15023 the shard sync, or is 0, the shard replica is ready to serve data and
15024 we can bring the node out of maintenance mode.
15025 Clear the target node’s maintenance mode
15027 You can now let the node start servicing data requests by putting
15028 "false" to the maintenance mode configuration endpoint, just as in step
15029 2.
15030 Verify that the node is not in maintenance mode by performing a GET
15032 /_up on that node’s individual endpoint.
15033 Finally, check that your load balancer has returned the node to the
15035 pool of available backend nodes.
15036 Update cluster metadata again to remove the source shard
15038 Now, remove the source shard from the shard map the same way that you
15039 added the new target shard to the shard map in step 2. Be sure to add
15040 the ["remove", {range}, {source-shard}] entry to the end of the
15041 changelog as well as modifying both the by_node and by_range sections
15042 of the database metadata document.
15043 Remove the shard and secondary index files from the source node
15045 Finally, you can remove the source shard replica by deleting its file
15046 from the command line on the source host, along with any view shard
15047 replicas:
15048 $ rm {couch-dir}/data/shards/{range}/{db}.{datecode}.couch
15050 $ rm -r {couch-dir}/data/.shards/{range}/{db}.{datecode}*
15051 Congratulations! You have moved a database shard replica. By adding and
15053 removing database shard replicas in this way, you can change the clus‐
15054 ter’s shard layout, also known as a shard map.
15055 Specifying database placement
15057 You can configure CouchDB to put shard replicas on certain nodes at
15058 database creation time using placement rules.
15059 WARNING:
15061 Use of the placement option will override the n option, both in the
15062 .ini file as well as when specified in a URL.
15063 First, each node must be labeled with a zone attribute. This defines
15065 which zone each node is in. You do this by editing the node’s document
15066 in the special /_nodes database, which is accessed through the special
15067 node-local API endpoint at /_node/_local/_nodes/{node-name}. Add a key
15068 value pair of the form:
15069 "zone": "{zone-name}"
15071 Do this for all of the nodes in your cluster. For example:
15073 $ curl -X PUT http://localhost/_node/_local/_nodes/{node-name} \
15075 -d '{ \
15076 "_id": "{node-name}",
15077 "_rev": "{rev}",
15078 "zone": "{zone-name}"
15079 }'
15080 In the local config file (local.ini) of each node, define a consistent
15082 cluster-wide setting like:
15083 [cluster]
15085 placement = {zone-name-1}:2,{zone-name-2}:1
15086 In this example, CouchDB will ensure that two replicas for a shard will
15088 be hosted on nodes with the zone attribute set to {zone-name-1} and one
15089 replica will be hosted on a new with the zone attribute set to
15090 {zone-name-2}.
15091 This approach is flexible, since you can also specify zones on a per-
15093 database basis by specifying the placement setting as a query parameter
15094 when the database is created, using the same syntax as the ini file:
15095 curl -X PUT $COUCH_URL:5984/{db}?zone={zone}
15097 The placement argument may also be specified. Note that this will over‐
15099 ride the logic that determines the number of created replicas!
15100 Note that you can also use this system to ensure certain nodes in the
15102 cluster do not host any replicas for newly created databases, by giving
15103 them a zone attribute that does not appear in the [cluster] placement
15104 string.
15105 Splitting Shards
15107 The api/server/reshard is an HTTP API for shard manipulation. Currently
15108 it only supports shard splitting. To perform shard merging, refer to
15109 the manual process outlined in the Merging Shards section.
15110 The main way to interact with api/server/reshard is to create reshard‐
15112 ing jobs, monitor those jobs, wait until they complete, remove them,
15113 post new jobs, and so on. What follows are a few steps one might take
15114 to use this API to split shards.
15115 At first, it’s a good idea to call GET /_reshard to see a summary of
15117 resharding on the cluster.
15118 $ curl -s $COUCH_URL:5984/_reshard | jq .
15120 {
15121 "state": "running",
15122 "state_reason": null,
15123 "completed": 3,
15124 "failed": 0,
15125 "running": 0,
15126 "stopped": 0,
15127 "total": 3
15128 }
15129 Two important things to pay attention to are the total number of jobs
15131 and the state.
15132 The state field indicates the state of resharding on the cluster. Nor‐
15134 mally it would be running, however, another user could have disabled
15135 resharding temporarily. Then, the state would be stopped and hopefully,
15136 there would be a reason or a comment in the value of the state_reason
15137 field. See Stopping Resharding Jobs for more details.
15138 The total number of jobs is important to keep an eye on because there
15140 is a maximum number of resharding jobs per node, and creating new jobs
15141 after the limit has been reached will result in an error. Before star‐
15142 ing new jobs it’s a good idea to remove already completed jobs. See
15143 reshard configuration section for the default value of max_jobs parame‐
15144 ter and how to adjust if needed.
15145 For example, if the jobs have completed, to remove all the jobs run:
15147 $ curl -s $COUCH_URL:5984/_reshard/jobs | jq -r '.jobs[].id' |\
15149 while read -r jobid; do\
15150 curl -s -XDELETE $COUCH_URL:5984/_reshard/jobs/$jobid\
15151 done
15152 Then it’s a good idea to see what the db shard map looks like.
15154 $ curl -s $COUCH_URL:5984/db1/_shards | jq '.'
15156 {
15157 "shards": {
15158 "00000000-7fffffff": [
15159 "node1@127.0.0.1",
15160 "node2@127.0.0.1",
15161 "node3@127.0.0.1"
15162 ],
15163 "80000000-ffffffff": [
15164 "node1@127.0.0.1",
15165 "node2@127.0.0.1",
15166 "node3@127.0.0.1"
15167 ]
15168 }
15169 }
15170 In this example we’ll split all the copies of the 00000000-7fffffff
15172 range. The API allows a combination of parameters such as: splitting
15173 all the ranges on all the nodes, all the ranges on just one node, or
15174 one particular range on one particular node. These are specified via
15175 the db, node and range job parameters.
15176 To split all the copies of 00000000-7fffffff we issue a request like
15178 this:
15179 $ curl -s -H "Content-type: application/json" -XPOST $COUCH_URL:5984/_reshard/jobs \
15181 -d '{"type": "split", "db":"db1", "range":"00000000-7fffffff"}' | jq '.'
15182 [
15183 {
15184 "ok": true,
15185 "id": "001-ef512cfb502a1c6079fe17e9dfd5d6a2befcc694a146de468b1ba5339ba1d134",
15186 "node": "node1@127.0.0.1",
15187 "shard": "shards/00000000-7fffffff/db1.1554242778"
15188 },
15189 {
15190 "ok": true,
15191 "id": "001-cec63704a7b33c6da8263211db9a5c74a1cb585d1b1a24eb946483e2075739ca",
15192 "node": "node2@127.0.0.1",
15193 "shard": "shards/00000000-7fffffff/db1.1554242778"
15194 },
15195 {
15196 "ok": true,
15197 "id": "001-fc72090c006d9b059d4acd99e3be9bb73e986d60ca3edede3cb74cc01ccd1456",
15198 "node": "node3@127.0.0.1",
15199 "shard": "shards/00000000-7fffffff/db1.1554242778"
15200 }
15201 ]
15202 The request returned three jobs, one job for each of the three copies.
15204 To check progress of these jobs use GET /_reshard/jobs or GET
15206 /_reshard/jobs/{jobid}.
15207 Eventually, these jobs should complete and the shard map should look
15209 like this:
15210 $ curl -s $COUCH_URL:5984/db1/_shards | jq '.'
15212 {
15213 "shards": {
15214 "00000000-3fffffff": [
15215 "node1@127.0.0.1",
15216 "node2@127.0.0.1",
15217 "node3@127.0.0.1"
15218 ],
15219 "40000000-7fffffff": [
15220 "node1@127.0.0.1",
15221 "node2@127.0.0.1",
15222 "node3@127.0.0.1"
15223 ],
15224 "80000000-ffffffff": [
15225 "node1@127.0.0.1",
15226 "node2@127.0.0.1",
15227 "node3@127.0.0.1"
15228 ]
15229 }
15230 }
15231 Stopping Resharding Jobs
15233 Resharding at the cluster level could be stopped and then restarted.
15234 This can be helpful to allow external tools which manipulate the shard
15235 map to avoid interfering with resharding jobs. To stop all resharding
15236 jobs on a cluster issue a PUT to /_reshard/state endpoint with the
15237 "state": "stopped" key and value. You can also specify an optional note
15238 or reason for stopping.
15239 For example:
15241 $ curl -s -H "Content-type: application/json" \
15243 -XPUT $COUCH_URL:5984/_reshard/state \
15244 -d '{"state": "stopped", "reason":"Moving some shards"}'
15245 {"ok": true}
15246 This state will then be reflected in the global summary:
15248 $ curl -s $COUCH_URL:5984/_reshard | jq .
15250 {
15251 "state": "stopped",
15252 "state_reason": "Moving some shards",
15253 "completed": 74,
15254 "failed": 0,
15255 "running": 0,
15256 "stopped": 0,
15257 "total": 74
15258 }
15259 To restart, issue a PUT request like above with running as the state.
15261 That should resume all the shard splitting jobs since their last check‐
15262 point.
15263 See the API reference for more details: api/server/reshard.
15265 Merging Shards
15267 The q value for a database can be set when the database is created or
15268 it can be increased later by splitting some of the shards Splitting
15269 Shards. In order to decrease q and merge some shards together, the
15270 database must be regenerated. Here are the steps:
15271 1. If there are running shard splitting jobs on the cluster, stop them
15273 via the HTTP API Stopping Resharding Jobs.
15274 2. Create a temporary database with the desired shard settings, by
15276 specifying the q value as a query parameter during the PUT opera‐
15277 tion.
15278 3. Stop clients accessing the database.
15280 4. Replicate the primary database to the temporary one. Multiple repli‐
15282 cations may be required if the primary database is under active use.
15283 5. Delete the primary database. Make sure nobody is using it!
15285 6. Recreate the primary database with the desired shard settings.
15287 7. Clients can now access the database again.
15289 8. Replicate the temporary back to the primary.
15291 9. Delete the temporary database.
15293 Once all steps have completed, the database can be used again. The
15295 cluster will create and distribute its shards according to placement
15296 rules automatically.
15297 Downtime can be avoided in production if the client application(s) can
15299 be instructed to use the new database instead of the old one, and a
15300 cut- over is performed during a very brief outage window.
15301 Clustered Purge
15303 The primary purpose of clustered purge is to clean databases that have
15304 multiple deleted tombstones or single documents that contain large num‐
15305 bers of conflicts. But it can also be used to purge any document
15306 (deleted or non-deleted) with any number of revisions.
15307 Clustered purge is designed to maintain eventual consistency and pre‐
15309 vent unnecessary invalidation of secondary indexes. For this, every
15310 database keeps track of a certain number of historical purges requested
15311 in the database, as well as its current purge_seq. Internal replica‐
15312 tions and secondary indexes process database’s purges and periodically
15313 update their corresponding purge checkpoint documents to report
15314 purge_seq processed by them. To ensure eventual consistency, the data‐
15315 base will remove stored historical purge requests only after they have
15316 been processed by internal replication jobs and secondary indexes.
15317 Internal Structures
15319 To enable internal replication of purge information between nodes and
15320 secondary indexes, two internal purge trees were added to a database
15321 file to track historical purges.
15322 purge_tree: UUID -> {PurgeSeq, DocId, Revs}
15324 purge_seq_tree: PurgeSeq -> {UUID, DocId, Revs}
15325 Each interactive request to _purge API, creates an ordered set of pairs
15327 on increasing purge_seq and purge_request, where purge_request is a
15328 tuple that contains docid and list of revisions. For each purge_request
15329 uuid is generated. A purge request is added to internal purge trees: a
15330 tuple {UUID -> {PurgeSeq, DocId, Revs}} is added to purge_tree, a tuple
15331 is {PurgeSeq -> {UUID, DocId, Revs}} added to purge_seq_tree.
15332 Compaction of Purges
15334 During the compaction of the database the oldest purge requests are to
15335 be removed to store only purged_infos_limit number of purges in the
15336 database. But in order to keep the database consistent with indexes
15337 and other replicas, we can only remove purge requests that have already
15338 been processed by indexes and internal replications jobs. Thus, occa‐
15339 sionally purge trees may store more than purged_infos_limit purges. If
15340 the number of stored purges in the database exceeds purged_infos_limit
15341 by a certain threshold, a warning is produced in logs signaling a prob‐
15342 lem of synchronization of database’s purges with indexes and other
15343 replicas.
15344 Local Purge Checkpoint Documents
15346 Indexes and internal replications of the database with purges create
15347 and periodically update local checkpoint purge documents:
15348 _local/purge-$type-$hash. These documents report the last purge_seq
15349 processed by them and the timestamp of the last processing. An example
15350 of a local checkpoint purge document:
15351 {
15353 "_id": "_local/purge-mrview-86cacdfbaf6968d4ebbc324dd3723fe7",
15354 "type": "mrview",
15355 "purge_seq": 10,
15356 "updated_on": 1540541874,
15357 "ddoc_id": "_design/foo",
15358 "signature": "5d10247925f826ae3e00966ec24b7bf6"
15359 }
15360 The below image shows possible local checkpoint documents that a data‐
15362 base may have.
15363 [image: Local Purge Checkpoint Documents] [image] Local Purge Check‐
15364 point Documents.UNINDENT
15365 Internal Replication
15367 Purge requests are replayed across all nodes in an eventually consis‐
15368 tent manner. Internal replication of purges consists of two steps:
15369 1. Pull replication. Internal replication first starts by pulling
15371 purges from target and applying them on source to make sure we don’t
15372 reintroduce to target source’s docs/revs that have been already purged
15373 on target. In this step, we use purge checkpoint documents stored on
15374 target to keep track of the last target’s purge_seq processed by the
15375 source. We find purge requests occurred after this purge_seq, and
15376 replay them on source. This step is done by updating the target’s
15377 checkpoint purge documents with the latest process purge_seq and time‐
15378 stamp.
15379 2. Push replication. Then internal replication proceeds as usual with
15381 an extra step inserted to push source’s purge requests to target. In
15382 this step, we use local internal replication checkpoint documents, that
15383 are updated both on target and source.
15384 Under normal conditions, an interactive purge request is already sent
15386 to every node containing a database shard’s replica, and applied on
15387 every replica. Internal replication of purges between nodes is just an
15388 extra step to ensure consistency between replicas, where all purge
15389 requests on one node are replayed on another node. In order not to
15390 replay the same purge request on a replica, each interactive purge
15391 request is tagged with a unique uuid. Internal replication filters out
15392 purge requests with UUIDs that already exist in the replica’s
15393 purge_tree, and applies only purge requests with UUIDs that don’t exist
15394 in the purge_tree. This is the reason why we needed to have two inter‐
15395 nal purge trees: 1) purge_tree: {UUID -> {PurgeSeq, DocId, Revs}}
15396 allows to quickly find purge requests with UUIDs that already exist in
15397 the replica; 2) purge_seq_tree: {PurgeSeq -> {UUID, DocId, Revs}}
15398 allows to iterate from a given purge_seq to collect all purge requests
15399 happened after this purge_seq.
15400 Indexes
15402 Each purge request will bump up update_seq of the database, so that
15403 each secondary index is also updated in order to apply the purge
15404 requests to maintain consistency within the main database.
15405 Config Settings
15407 These settings can be updated in the default.ini or local.ini:
15408 ┌──────────────────────┬─────────────────────┬─────────┐
15410 │Field │ Description │ Default │
15411 ├──────────────────────┼─────────────────────┼─────────┤
15412 │max_docu‐ │ Allowed maximum │ 100 │
15413 │ment_id_number │ number of documents │ │
15414 │ │ in one purge │ │
15415 │ │ request │ │
15416 ├──────────────────────┼─────────────────────┼─────────┤
15417 │max_revisions_num‐ │ Allowed maximum │ 1000 │
15418 │ber │ number of accumu‐ │ │
15419 │ │ lated revisions in │ │
15420 │ │ one purge request │ │
15421 ├──────────────────────┼─────────────────────┼─────────┤
15422 │allowed_purge_seq_lag │ Beside │ 100 │
15423 │ │ purged_infos_limit, │ │
15424 │ │ allowed additional │ │
15425 │ │ buffer to store │ │
15426 │ │ purge requests │ │
15427 ├──────────────────────┼─────────────────────┼─────────┤
15428 │index_lag_warn_sec‐ │ Allowed durations │ 86400 │
15429 │onds │ when index is not │ │
15430 │ │ updated for local │ │
15431 │ │ purge checkpoint │ │
15432 │ │ document │ │
15433 └──────────────────────┴─────────────────────┴─────────┘
15434 During a database compaction, we check all checkpoint purge docs. A
15436 client (an index or internal replication job) is allowed to have the
15437 last reported purge_seq to be smaller than the current database shard’s
15438 purge_seq by the value of (purged_infos_limit + allowed_purge_seq_lag).
15439 If the client’s purge_seq is even smaller, and the client has not
15440 checkpointed within index_lag_warn_seconds, it prevents compaction of
15441 purge trees and we have to issue the following log warning for this
15442 client:
15443 Purge checkpoint '_local/purge-mrview-9152d15c12011288629bcffba7693fd4’
15445 not updated in 86400 seconds in
15446 <<"shards/00000000-1fffffff/testdb12.1491979089">>
15447 If this type of log warning occurs, check the client to see why the
15449 processing of purge requests is stalled in it.
15450 There is a mapping relationship between a design document of indexes
15452 and local checkpoint docs. If a design document of indexes is updated
15453 or deleted, the corresponding local checkpoint document should be also
15454 automatically deleted. But in an unexpected case, when a design doc
15455 was updated/deleted, but its checkpoint document still exists in a
15456 database, the following warning will be issued:
15457 "Invalid purge doc '<<"_design/bar">>' on database
15459 <<"shards/00000000-1fffffff/testdb12.1491979089">>
15460 with purge_seq '50'"
15461 If this type of log warning occurs, remove the local purge doc from a
15463 database.
15464
15466 Compaction
15467 The compaction operation is the way to reduce disk space usage by
15468 removing unused and old data from database or view index files. This
15469 operation is very similar to the vacuum (SQLite ex.) operation avail‐
15470 able for other database management systems.
15471 During compaction of the target CouchDB creates new file with the .com‐
15473 pact extension and transfers only actual data into. Because of this,
15474 CouchDB checks first for the available disk space - it should be twice
15475 greater than the compacted file’s data.
15476 When all actual data is successfully transferred to the compacted file
15478 CouchDB replaces the target with the compacted file.
15479 Database Compaction
15481 Database compaction compresses the database file by removing unused
15482 file sections created during updates. Old documents revisions are
15483 replaced with small amount of metadata called tombstone which are used
15484 for conflicts resolution during replication. The number of stored revi‐
15485 sions (and their tombstones) can be configured by using the _revs_limit
15486 URL endpoint.
15487 Compaction can be manually triggered per database and runs as a back‐
15489 ground task. To start it for specific database there is need to send
15490 HTTP POST /{db}/_compact sub-resource of the target database:
15491 curl -H "Content-Type: application/json" -X POST http://localhost:5984/my_db/_compact
15493 On success, HTTP status 202 Accepted is returned immediately:
15495 HTTP/1.1 202 Accepted
15497 Cache-Control: must-revalidate
15498 Content-Length: 12
15499 Content-Type: text/plain; charset=utf-8
15500 Date: Wed, 19 Jun 2013 09:43:52 GMT
15501 Server: CouchDB (Erlang/OTP)
15502 {"ok":true}
15504 Although the request body is not used you must still specify
15506 Content-Type header with application/json value for the request. If you
15507 don’t, you will be aware about with HTTP status 415 Unsupported Media
15508 Type response:
15509 HTTP/1.1 415 Unsupported Media Type
15511 Cache-Control: must-revalidate
15512 Content-Length: 78
15513 Content-Type: application/json
15514 Date: Wed, 19 Jun 2013 09:43:44 GMT
15515 Server: CouchDB (Erlang/OTP)
15516 {"error":"bad_content_type","reason":"Content-Type must be application/json"}
15518 When the compaction is successful started and running it is possible to
15520 get information about it via database information resource:
15521 curl http://localhost:5984/my_db
15523 HTTP/1.1 200 OK
15525 Cache-Control: must-revalidate
15526 Content-Length: 246
15527 Content-Type: application/json
15528 Date: Wed, 19 Jun 2013 16:51:20 GMT
15529 Server: CouchDB (Erlang/OTP)
15530 {
15532 "committed_update_seq": 76215,
15533 "compact_running": true,
15534 "db_name": "my_db",
15535 "disk_format_version": 6,
15536 "doc_count": 5091,
15537 "doc_del_count": 0,
15538 "instance_start_time": "0",
15539 "purge_seq": 0,
15540 "sizes": {
15541 "active": 3787996,
15542 "disk": 17703025,
15543 "external": 4763321
15544 },
15545 "update_seq": 76215
15546 }
15547 Note that compaction_running field is true indicating that compaction
15549 is actually running. To track the compaction progress you may query the
15550 _active_tasks resource:
15551 curl http://localhost:5984/_active_tasks
15553 HTTP/1.1 200 OK
15555 Cache-Control: must-revalidate
15556 Content-Length: 175
15557 Content-Type: application/json
15558 Date: Wed, 19 Jun 2013 16:27:23 GMT
15559 Server: CouchDB (Erlang/OTP)
15560 [
15562 {
15563 "changes_done": 44461,
15564 "database": "my_db",
15565 "pid": "<0.218.0>",
15566 "progress": 58,
15567 "started_on": 1371659228,
15568 "total_changes": 76215,
15569 "type": "database_compaction",
15570 "updated_on": 1371659241
15571 }
15572 ]
15573 Views Compaction
15575 Views are also need compaction like databases, unlike databases views
15576 are compacted by groups per design document. To start their compaction
15577 there is need to send HTTP POST /{db}/_compact/{ddoc} request:
15578 curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_compact/designname
15580 {"ok":true}
15582 This compacts the view index from the current version of the specified
15584 design document. The HTTP response code is 202 Accepted (like
15585 compaction for databases) and a compaction background task will be cre‐
15586 ated.
15587 Views cleanup
15589 View indexes on disk are named after their MD5 hash of the view defini‐
15590 tion. When you change a view, old indexes remain on disk. To clean up
15591 all outdated view indexes (files named after the MD5 representation of
15592 views, that does not exist anymore) you can trigger a view cleanup:
15593 curl -H "Content-Type: application/json" -X POST http://localhost:5984/dbname/_view_cleanup
15595 {"ok":true}
15597 Automatic Compaction
15599 CouchDB’s automatic compaction daemon, internally known as “smoosh”,
15600 will trigger compaction jobs for both databases and views based on con‐
15601 figurable thresholds for the sparseness of a file and the total amount
15602 of space that can be recovered.
15603 Channels
15605 Smoosh works using the concept of channels. A channel is essentially a
15606 queue of pending compactions. There are separate sets of active chan‐
15607 nels for databases and views. Each channel is assigned a configuration
15608 which defines whether a compaction ends up in the channel’s queue and
15609 how compactions are prioritized within that queue.
15610 Smoosh takes each channel and works through the compactions queued in
15612 each in priority order. Each channel is processed concurrently, so the
15613 priority levels only matter within a given channel. Each channel has an
15614 assigned number of active compactions, which defines how many com‐
15615 pactions happen for that channel in parallel. For example, a cluster
15616 with a lot of database churn but few views might require more active
15617 compactions in the database channel(s).
15618 It’s important to remember that a channel is local to a CouchDB node;
15620 that is, each node maintains and processes an independent set of com‐
15621 pactions. Channels are defined as either “ratio” channels or “slack”
15622 channels, depending on the type of algorithm used for prioritization:
15623 · Ratio: uses the ratio of sizes.file / sizes.active as its driving
15625 calculation. The result X must be greater than some configurable
15626 value Y for a compaction to be added to the queue. Compactions are
15627 then prioritised for higher values of X.
15628 · Slack: uses the difference of sizes.file - sizes.active as its driv‐
15630 ing calculation. The result X must be greater than some configurable
15631 value Y for a compaction to be added to the queue. Compactions are
15632 prioritised for higher values of X.
15633 In both cases, Y is set using the min_priority configuration variable.
15635 CouchDB ships with four channels pre-configured: one channel of each
15636 type for databases, and another one for views.
15637 Channel Configuration
15639 Channels are defined using [smoosh.<channel_name>] configuration
15640 blocks, and activated by naming the channel in the db_channels or
15641 view_channels configuration setting in the [smoosh] block. The default
15642 configuration is
15643 [smoosh]
15645 db_channels = upgrade_dbs,ratio_dbs,slack_dbs
15646 view_channels = upgrade_views,ratio_views,slack_views
15647 [smoosh.ratio_dbs]
15649 priority = ratio
15650 min_priority = 5.0
15651 [smoosh.ratio_views]
15653 priority = ratio
15654 min_priority = 5.0
15655 [smoosh.slack_dbs]
15657 priority = slack
15658 min_priority = 16777216
15659 [smoosh.slack_views]
15661 priority = slack
15662 min_priority = 16777216
15663 The “upgrade” channels are a special pair of channels that only check
15665 whether the disk_format_version for the file matches the current ver‐
15666 sion, and enqueue the file for compaction (which has the side effect of
15667 upgrading the file format) if that’s not the case. There are several
15668 additional properties that can be configured for each channel; these
15669 are documented in the configuration API
15670 Scheduling Windows
15672 Each compaction channel can be configured to run only during certain
15673 hours of the day. The channel-specific from, to, and strict_window con‐
15674 figuration settings control this behavior. For example
15675 [smoosh.overnight_channel]
15677 from = 20:00
15678 to = 06:00
15679 strict_window = true
15680 The strict_window setting will cause the compaction daemon to suspend
15682 all active compactions in this channel when exiting the window, and
15683 resume them when re-entering. If strict_window is left at its default
15684 of false, the active compactions will be allowed to complete but no new
15685 compactions will be started.
15686 Migration Guide
15688 Previous versions of CouchDB shipped with a simpler compaction daemon.
15689 The configuration system for the new daemon is not backwards-compatible
15690 with the old one, so users with customized compaction configurations
15691 will need to port them to the new setup. The old daemon’s compaction
15692 rules configuration looked like
15693 [compaction_daemon]
15695 min_file_size = 131072
15696 check_interval = 3600
15697 snooze_period_ms = 3000
15698 [compactions]
15700 mydb = [{db_fragmentation, "70%"}, {view_fragmentation, "60%"}, {parallel_view_compaction, true}]
15701 _default = [{db_fragmentation, "50%"}, {view_fragmentation, "55%"}, {from, "20:00"}, {to, "06:00"}, {strict_window, true}]
15702 Many of the elements of this configuration can be ported over to the
15704 new system. Examining each in detail:
15705 · min_file_size is now configured on a per-channel basis using the
15707 min_size config setting.
15708 · db_fragmentation is equivalent to configuring a priority = ratio
15710 channel with min_priority set to 1.0 / (1 - db_fragmentation/100) and
15711 then listing that channel in the [smoosh] db_channels config setting.
15712 · view_fragmention is likewise equivalent to configuring a priority =
15714 ratio channel with min_priority set to 1.0 / (1 - view_fragmenta‐
15715 tion/100) and then listing that channel in the [smoosh] view_channels
15716 config setting.
15717 · from / to / strict_window: each of these settings can be applied on a
15719 per-channel basis in the new daemon. The one behavior change is that
15720 the new daemon will suspend compactions upon exiting the allowed win‐
15721 dow instead of canceling them outright, and resume them when
15722 re-entering.
15723 · parallel_view_compaction: each compaction channel has a concurrency
15725 setting that controls how many compactions will execute in parallel
15726 in that channel. The total parallelism is the sum of the concurrency
15727 settings of all active channels. This is a departure from the previ‐
15728 ous behavior, in which the daemon would only focus on one database
15729 and/or its views (depending on the value of this flag) at a time.
15730 The check_interval and snooze_period_ms settings are obsolete in the
15732 event-driven design of the new daemon. The new daemon does not support
15733 setting database-specific thresholds as in the mydb setting above.
15734 Rather, channels can be configured to focus on specific classes of
15735 files: large databases, small view indexes, and so on. Most cases of
15736 named database compaction rules can be expressed using properties of
15737 those databases and/or their associated views.
15738 Performance
15740 With up to tens of thousands of documents you will generally find
15741 CouchDB to perform well no matter how you write your code. Once you
15742 start getting into the millions of documents you need to be a lot more
15743 careful.
15744 Disk I/O
15746 File Size
15747 The smaller your file size, the less I/O operations there will be, the
15748 more of the file can be cached by CouchDB and the operating system, the
15749 quicker it is to replicate, backup etc. Consequently you should care‐
15750 fully examine the data you are storing. For example it would be silly
15751 to use keys that are hundreds of characters long, but your program
15752 would be hard to maintain if you only used single character keys. Care‐
15753 fully consider data that is duplicated by putting it in views.
15754 Disk and File System Performance
15756 Using faster disks, striped RAID arrays and modern file systems can all
15757 speed up your CouchDB deployment. However, there is one option that can
15758 increase the responsiveness of your CouchDB server when disk perfor‐
15759 mance is a bottleneck. From the Erlang documentation for the file mod‐
15760 ule:
15761 On operating systems with thread support, it is possible to let file
15762 operations be performed in threads of their own, allowing other
15763 Erlang processes to continue executing in parallel with the file
15764 operations. See the command line flag +A in erl(1).
15765 Setting this argument to a number greater than zero can keep your
15767 CouchDB installation responsive even during periods of heavy disk uti‐
15768 lization. The easiest way to set this option is through the ERL_FLAGS
15769 environment variable. For example, to give Erlang four threads with
15770 which to perform I/O operations add the following to (pre‐
15771 fix)/etc/defaults/couchdb (or equivalent):
15772 export ERL_FLAGS="+A 4"
15774 System Resource Limits
15776 One of the problems that administrators run into as their deployments
15777 become large are resource limits imposed by the system and by the
15778 application configuration. Raising these limits can allow your deploy‐
15779 ment to grow beyond what the default configuration will support.
15780 CouchDB Configuration Options
15782 max_dbs_open
15783 In your configuration (local.ini or similar) familiarize yourself with
15784 the couchdb/max_dbs_open:
15785 [couchdb]
15787 max_dbs_open = 100
15788 This option places an upper bound on the number of databases that can
15790 be open at one time. CouchDB reference counts database accesses inter‐
15791 nally and will close idle databases when it must. Sometimes it is nec‐
15792 essary to keep more than the default open at once, such as in deploy‐
15793 ments where many databases will be continuously replicating.
15794 Erlang
15796 Even if you’ve increased the maximum connections CouchDB will allow,
15797 the Erlang runtime system will not allow more than 1024 connections by
15798 default. Adding the following directive to (prefix)/etc/default/couchdb
15799 (or equivalent) will increase this limit (in this case to 4096):
15800 export ERL_MAX_PORTS=4096
15802 CouchDB versions up to 1.1.x also create Erlang Term Storage (ETS)
15804 tables for each replication. If you are using a version of CouchDB
15805 older than 1.2 and must support many replications, also set the
15806 ERL_MAX_ETS_TABLES variable. The default is approximately 1400 tables.
15807 Note that on Mac OS X, Erlang will not actually increase the file
15809 descriptor limit past 1024 (i.e. the system header–defined value of
15810 FD_SETSIZE). See this tip for a possible workaround and this thread for
15811 a deeper explanation.
15812 Maximum open file descriptors (ulimit)
15814 Most *nix operating systems impose various resource limits on every
15815 process. The method of increasing these limits varies, depending on
15816 your init system and particular OS release. The default value for many
15817 OSes is 1024 or 4096. On a system with many databases or many views,
15818 CouchDB can very rapidly hit this limit.
15819 If your system is set up to use the Pluggable Authentication Modules (‐
15821 PAM) system (as is the case with nearly all modern Linuxes), increasing
15822 this limit is straightforward. For example, creating a file named
15823 /etc/security/limits.d/100-couchdb.conf with the following contents
15824 will ensure that CouchDB can open up to 10000 file descriptors at once:
15825 #<domain> <type> <item> <value>
15827 couchdb hard nofile 10000
15828 couchdb soft nofile 10000
15829 If you are using our Debian/Ubuntu sysvinit script
15831 (/etc/init.d/couchdb), you also need to raise the limits for the root
15832 user:
15833 #<domain> <type> <item> <value>
15835 root hard nofile 10000
15836 root soft nofile 10000
15837 You may also have to edit the /etc/pam.d/common-session and
15839 /etc/pam.d/common-session-noninteractive files to add the line:
15840 session required pam_limits.so
15842 if it is not already present.
15844 For systemd-based Linuxes (such as CentOS/RHEL 7, Ubuntu 16.04+, Debian
15846 8 or newer), assuming you are launching CouchDB from systemd, you must
15847 also override the upper limit by creating the file /etc/systemd/sys‐
15848 tem/<servicename>.d/override.conf with the following content:
15849 [Service]
15851 LimitNOFILE=#######
15852 and replacing the ####### with the upper limit of file descriptors
15854 CouchDB is allowed to hold open at once.
15855 If your system does not use PAM, a ulimit command is usually available
15857 for use in a custom script to launch CouchDB with increased resource
15858 limits. Typical syntax would be something like ulimit -n 10000.
15859 In general, modern UNIX-like systems can handle very large numbers of
15861 file handles per process (e.g. 100000) without problem. Don’t be afraid
15862 to increase this limit on your system.
15863 Network
15865 There is latency overhead making and receiving each request/response.
15866 In general you should do your requests in batches. Most APIs have some
15867 mechanism to do batches, usually by supplying lists of documents or
15868 keys in the request body. Be careful what size you pick for the
15869 batches. The larger batch requires more time your client has to spend
15870 encoding the items into JSON and more time is spent decoding that num‐
15871 ber of responses. Do some benchmarking with your own configuration and
15872 typical data to find the sweet spot. It is likely to be between one
15873 and ten thousand documents.
15874 If you have a fast I/O system then you can also use concurrency - have
15876 multiple requests/responses at the same time. This mitigates the
15877 latency involved in assembling JSON, doing the networking and decoding
15878 JSON.
15879 As of CouchDB 1.1.0, users often report lower write performance of doc‐
15881 uments compared to older releases. The main reason is that this release
15882 ships with the more recent version of the HTTP server library MochiWeb,
15883 which by default sets the TCP socket option SO_NODELAY to false. This
15884 means that small data sent to the TCP socket, like the reply to a docu‐
15885 ment write request (or reading a very small document), will not be sent
15886 immediately to the network - TCP will buffer it for a while hoping that
15887 it will be asked to send more data through the same socket and then
15888 send all the data at once for increased performance. This TCP buffer‐
15889 ing behaviour can be disabled via httpd/socket_options:
15890 [httpd]
15892 socket_options = [{nodelay, true}]
15893 SEE ALSO:
15895 Bulk load and store API.
15896 Connection limit
15898 MochiWeb handles CouchDB requests. The default maximum number of con‐
15899 nections is 2048. To change this limit, use the server_options configu‐
15900 ration variable. max indicates maximum number of connections.
15901 [chttpd]
15903 server_options = [{backlog, 128}, {acceptor_pool_size, 16}, {max, 4096}]
15904 CouchDB
15906 DELETE operation
15907 When you DELETE a document the database will create a new revision
15908 which contains the _id and _rev fields as well as the _deleted flag.
15909 This revision will remain even after a database compaction so that the
15910 deletion can be replicated. Deleted documents, like non-deleted docu‐
15911 ments, can affect view build times, PUT and DELETE request times, and
15912 the size of the database since they increase the size of the B+Tree.
15913 You can see the number of deleted documents in database information. If
15914 your use case creates lots of deleted documents (for example, if you
15915 are storing short-term data like log entries, message queues, etc), you
15916 might want to periodically switch to a new database and delete the old
15917 one (once the entries in it have all expired).
15918 Document’s ID
15920 The db file size is derived from your document and view sizes but also
15921 on a multiple of your _id sizes. Not only is the _id present in the
15922 document, but it and parts of it are duplicated in the binary tree
15923 structure CouchDB uses to navigate the file to find the document in the
15924 first place. As a real world example for one user switching from 16
15925 byte ids to 4 byte ids made a database go from 21GB to 4GB with 10 mil‐
15926 lion documents (the raw JSON text when from 2.5GB to 2GB).
15927 Inserting with sequential (and at least sorted) ids is faster than ran‐
15929 dom ids. Consequently you should consider generating ids yourself,
15930 allocating them sequentially and using an encoding scheme that consumes
15931 fewer bytes. For example, something that takes 16 hex digits to repre‐
15932 sent can be done in 4 base 62 digits (10 numerals, 26 lower case, 26
15933 upper case).
15934 Views
15936 Views Generation
15937 Views with the JavaScript query server are extremely slow to generate
15938 when there are a non-trivial number of documents to process. The gener‐
15939 ation process won’t even saturate a single CPU let alone your I/O. The
15940 cause is the latency involved in the CouchDB server and separate
15941 couchjs query server, dramatically indicating how important it is to
15942 take latency out of your implementation.
15943 You can let view access be “stale” but it isn’t practical to determine
15945 when that will occur giving you a quick response and when views will be
15946 updated which will take a long time. (A 10 million document database
15947 took about 10 minutes to load into CouchDB but about 4 hours to do view
15948 generation).
15949 In a cluster, “stale” requests are serviced by a fixed set of shards in
15951 order to present users with consistent results between requests. This
15952 comes with an availability trade-off - the fixed set of shards might
15953 not be the most responsive / available within the cluster. If you don’t
15954 need this kind of consistency (e.g. your indexes are relatively
15955 static), you can tell CouchDB to use any available replica by specify‐
15956 ing stable=false&update=false instead of stale=ok, or sta‐
15957 ble=false&update=lazy instead of stale=update_after.
15958 View information isn’t replicated - it is rebuilt on each database so
15960 you can’t do the view generation on a separate sever.
15961 Built-In Reduce Functions
15963 If you’re using a very simple view function that only performs a sum or
15964 count reduction, you can call native Erlang implementations of them by
15965 simply writing _sum or _count in place of your function declaration.
15966 This will speed up things dramatically, as it cuts down on IO between
15967 CouchDB and the JavaScript query server. For example, as mentioned on
15968 the mailing list, the time for outputting an (already indexed and
15969 cached) view with about 78,000 items went down from 60 seconds to 4
15970 seconds.
15971 Before:
15973 {
15975 "_id": "_design/foo",
15976 "views": {
15977 "bar": {
15978 "map": "function (doc) { emit(doc.author, 1); }",
15979 "reduce": "function (keys, values, rereduce) { return sum(values); }"
15980 }
15981 }
15982 }
15983 After:
15985 {
15987 "_id": "_design/foo",
15988 "views": {
15989 "bar": {
15990 "map": "function (doc) { emit(doc.author, 1); }",
15991 "reduce": "_sum"
15992 }
15993 }
15994 }
15995 SEE ALSO:
15997 reducefun/builtin
15998 Backing up CouchDB
16000 CouchDB has three different types of files it can create during run‐
16001 time:
16002 · Database files (including secondary indexes)
16004 · Configuration files (*.ini)
16006 · Log files (if configured to log to disk)
16008 Below are strategies for ensuring consistent backups of all of these
16010 files.
16011 Database Backups
16013 The simplest and easiest approach for CouchDB backup is to use CouchDB
16014 replication to another CouchDB installation. You can choose between
16015 normal (one-shot) or continuous replications depending on your need.
16016 However, you can also copy the actual .couch files from the CouchDB
16018 data directory (by default, data/) at any time, without problem.
16019 CouchDB’s append-only storage format for both databases and secondary
16020 indexes ensures that this will work without issue.
16021 To ensure reliability of backups, it is recommended that you back up
16023 secondary indexes (stored under data/.shards) prior to backing up the
16024 main database files (stored under data/shards as well as the sys‐
16025 tem-level databases at the parent data/ directory). This is because
16026 CouchDB will automatically handle views/secondary indexes that are
16027 slightly out of date by updating them on the next read access, but
16028 views or secondary indexes that are newer than their associated data‐
16029 bases will trigger a full rebuild of the index. This can be a very
16030 costly and time-consuming operation, and can impact your ability to
16031 recover quickly in a disaster situation.
16032 On supported operating systems/storage environments, you can also make
16034 use of storage snapshots. These have the advantage of being
16035 near-instantaneous when working with block storage systems such as ZFS
16036 or LVM or Amazon EBS. When using snapshots at the block-storage level,
16037 be sure to quiesce the file system with an OS-level utility such as
16038 Linux’s fsfreeze if necessary. If unsure, consult your operating sys‐
16039 tem’s or cloud provider’s documentation for more detail.
16040 Configuration Backups
16042 CouchDB’s configuration system stores data in .ini files under the con‐
16043 figuration directory (by default, etc/). If changes are made to the
16044 configuration at runtime, the very last file in the configuration chain
16045 will be updated with the changes.
16046 Simple back up the entire etc/ directory to ensure a consistent config‐
16048 uration after restoring from backup.
16049 If no changes to the configuration are made at runtime through the HTTP
16051 API, and all configuration files are managed by a configuration manage‐
16052 ment system (such as Ansible or Chef), there is no need to backup the
16053 configuration directory.
16054 Log Backups
16056 If configured to log to a file, you may want to back up the log files
16057 written by CouchDB. Any backup solution for these files works.
16058 Under UNIX-like systems, if using log rotation software, a
16060 copy-then-truncate approach is necessary. This will truncate the origi‐
16061 nal log file to zero size in place after creating a copy. CouchDB does
16062 not recognize any signal to be told to close its log file and create a
16063 new one. Because of this, and because of differences in how file han‐
16064 dles function, there is no straightforward log rotation solution under
16065 Microsoft Windows other than periodic restarts of the CouchDB process.
16066
16068 Fauxton Setup
16069 Fauxton is included with CouchDB 2.0, so make sure CouchDB is running,
16070 then go to:
16071 http://127.0.0.1:5984/_utils/
16073 You can also upgrade to the latest version of Fauxton by using npm:
16075 $ npm install -g fauxton
16077 $ fauxton
16078 (Recent versions of node.js and npm are required.)
16080 Fauxton Visual Guide
16082 You can find the Visual Guide here:
16083 http://couchdb.apache.org/fauxton-visual-guide
16084 Development Server
16086 Recent versions of node.js and npm are required.
16087 Using the dev server is the easiest way to use Fauxton, specially when
16089 developing for it:
16090 $ git clone https://github.com/apache/couchdb-fauxton.git
16092 $ npm install && npm run dev
16093 Understanding Fauxton Code layout
16095 Each bit of functionality is its own separate module or addon.
16096 All core modules are stored under app/module and any addons that are
16098 optional are under app/addons.
16099 We use backbone.js and Backbone.layoutmanager quite heavily, so best to
16101 get an idea how they work. Its best at this point to read through a
16102 couple of the modules and addons to get an idea of how they work.
16103 Two good starting points are app/addon/config and app/modules/data‐
16105 bases.
16106 Each module must have a base.js file, this is read and compile when
16108 Fauxton is deployed.
16109 The resource.js file is usually for your Backbone.Models and Back‐
16111 bone.Collections, view.js for your Backbone.Views.
16112 The routes.js is used to register a url path for your view along with
16114 what layout, data, breadcrumbs and api point is required for the view.
16115 ToDo items
16117 Checkout JIRA or GitHub Issues for a list of items to do.
16118
16120 This is a list of experimental features in CouchDB. They are included
16121 in a release because the development team is requesting feedback from
16122 the larger developer community. As such, please play around with these
16123 features and send us feedback, thanks!
16124 Use at your own risk! Do not rely on these features for critical appli‐
16126 cations.
16127 Content-Security-Policy (CSP) Header Support for /_utils (Fauxton)
16129 This will just work with Fauxton. You can enable it in your config: you
16130 can enable the feature in general and change the default header that is
16131 sent for everything in /_utils.
16132 [csp]
16134 enable = true
16135 Then restart CouchDB.
16137 Have fun!
16139
16141 The components of the API URL path help determine the part of the
16142 CouchDB server that is being accessed. The result is the structure of
16143 the URL request both identifies and effectively describes the area of
16144 the database you are accessing.
16145 As with all URLs, the individual components are separated by a forward
16147 slash.
16148 As a general rule, URL components and JSON fields starting with the _
16150 (underscore) character represent a special component or entity within
16151 the server or returned object. For example, the URL fragment /_all_dbs
16152 gets a list of all of the databases in a CouchDB instance.
16153 This reference is structured according to the URL structure, as below.
16155 API Basics
16157 The CouchDB API is the primary method of interfacing to a CouchDB
16158 instance. Requests are made using HTTP and requests are used to
16159 request information from the database, store new data, and perform
16160 views and formatting of the information stored within the documents.
16161 Requests to the API can be categorised by the different areas of the
16163 CouchDB system that you are accessing, and the HTTP method used to send
16164 the request. Different methods imply different operations, for example
16165 retrieval of information from the database is typically handled by the
16166 GET operation, while updates are handled by either a POST or PUT
16167 request. There are some differences between the information that must
16168 be supplied for the different methods. For a guide to the basic HTTP
16169 methods and request structure, see Request Format and Responses.
16170 For nearly all operations, the submitted data, and the returned data
16172 structure, is defined within a JavaScript Object Notation (JSON)
16173 object. Basic information on the content and data types for JSON are
16174 provided in JSON Basics.
16175 Errors when accessing the CouchDB API are reported using standard HTTP
16177 Status Codes. A guide to the generic codes returned by CouchDB are pro‐
16178 vided in HTTP Status Codes.
16179 When accessing specific areas of the CouchDB API, specific information
16181 and examples on the HTTP methods and request, JSON structures, and
16182 error codes are provided.
16183 Request Format and Responses
16185 CouchDB supports the following HTTP request methods:
16186 · GET
16188 Request the specified item. As with normal HTTP requests, the format
16190 of the URL defines what is returned. With CouchDB this can include
16191 static items, database documents, and configuration and statistical
16192 information. In most cases the information is returned in the form of
16193 a JSON document.
16194 · HEAD
16196 The HEAD method is used to get the HTTP header of a GET request with‐
16198 out the body of the response.
16199 · POST
16201 Upload data. Within CouchDB POST is used to set values, including
16203 uploading documents, setting document values, and starting certain
16204 administration commands.
16205 · PUT
16207 Used to put a specified resource. In CouchDB PUT is used to create
16209 new objects, including databases, documents, views and design docu‐
16210 ments.
16211 · DELETE
16213 Deletes the specified resource, including documents, views, and
16215 design documents.
16216 · COPY
16218 A special method that can be used to copy documents and objects.
16220 If you use an unsupported HTTP request type with an URL that does not
16222 support the specified type then a 405 - Method Not Allowed will be
16223 returned, listing the supported HTTP methods. For example:
16224 {
16226 "error":"method_not_allowed",
16227 "reason":"Only GET,HEAD allowed"
16228 }
16229 HTTP Headers
16231 Because CouchDB uses HTTP for all communication, you need to ensure
16232 that the correct HTTP headers are supplied (and processed on retrieval)
16233 so that you get the right format and encoding. Different environments
16234 and clients will be more or less strict on the effect of these HTTP
16235 headers (especially when not present). Where possible you should be as
16236 specific as possible.
16237 Request Headers
16239 · Accept
16240 Specifies the list of accepted data types to be returned by the
16242 server (i.e. that are accepted/understandable by the client). The
16243 format should be a list of one or more MIME types, separated by
16244 colons.
16245 For the majority of requests the definition should be for JSON data
16247 (application/json). For attachments you can either specify the MIME
16248 type explicitly, or use */* to specify that all file types are sup‐
16249 ported. If the Accept header is not supplied, then the */* MIME type
16250 is assumed (i.e. client accepts all formats).
16251 The use of Accept in queries for CouchDB is not required, but is
16253 highly recommended as it helps to ensure that the data returned can
16254 be processed by the client.
16255 If you specify a data type using the Accept header, CouchDB will
16257 honor the specified type in the Content-type header field returned.
16258 For example, if you explicitly request application/json in the Accept
16259 of a request, the returned HTTP headers will use the value in the
16260 returned Content-type field.
16261 For example, when sending a request without an explicit Accept
16263 header, or when specifying */*:
16264 GET /recipes HTTP/1.1
16266 Host: couchdb:5984
16267 Accept: */*
16268 The returned headers are:
16270 HTTP/1.1 200 OK
16272 Server: CouchDB (Erlang/OTP)
16273 Date: Thu, 13 Jan 2011 13:39:34 GMT
16274 Content-Type: text/plain;charset=utf-8
16275 Content-Length: 227
16276 Cache-Control: must-revalidate
16277 NOTE:
16279 The returned content type is text/plain even though the informa‐
16280 tion returned by the request is in JSON format.
16281 Explicitly specifying the Accept header:
16283 GET /recipes HTTP/1.1
16285 Host: couchdb:5984
16286 Accept: application/json
16287 The headers returned include the application/json content type:
16289 HTTP/1.1 200 OK
16291 Server: CouchDB (Erlang/OTP)
16292 Date: Thu, 13 Jan 2013 13:40:11 GMT
16293 Content-Type: application/json
16294 Content-Length: 227
16295 Cache-Control: must-revalidate
16296 · Content-type
16298 Specifies the content type of the information being supplied within
16300 the request. The specification uses MIME type specifications. For the
16301 majority of requests this will be JSON (application/json). For some
16302 settings the MIME type will be plain text. When uploading attachments
16303 it should be the corresponding MIME type for the attachment or binary
16304 (application/octet-stream).
16305 The use of the Content-type on a request is highly recommended.
16307 Response Headers
16309 Response headers are returned by the server when sending back content
16310 and include a number of different header fields, many of which are
16311 standard HTTP response header and have no significance to CouchDB oper‐
16312 ation. The list of response headers important to CouchDB are listed
16313 below.
16314 · Cache-control
16316 The cache control HTTP response header provides a suggestion for
16318 client caching mechanisms on how to treat the returned information.
16319 CouchDB typically returns the must-revalidate, which indicates that
16320 the information should be revalidated if possible. This is used to
16321 ensure that the dynamic nature of the content is correctly updated.
16322 · Content-length
16324 The length (in bytes) of the returned content.
16326 · Content-type
16328 Specifies the MIME type of the returned data. For most request, the
16330 returned MIME type is text/plain. All text is encoded in Unicode
16331 (UTF-8), and this is explicitly stated in the returned Content-type,
16332 as text/plain;charset=utf-8.
16333 · Etag
16335 The Etag HTTP header field is used to show the revision for a docu‐
16337 ment, or a view.
16338 ETags have been assigned to a map/reduce group (the collection of
16340 views in a single design document). Any change to any of the indexes
16341 for those views would generate a new ETag for all view URLs in a sin‐
16342 gle design doc, even if that specific view’s results had not changed.
16343 Each _view URL has its own ETag which only gets updated when changes
16345 are made to the database that effect that index. If the index for
16346 that specific view does not change, that view keeps the original ETag
16347 head (therefore sending back 304 - Not Modified more often).
16348 · Transfer-Encoding
16350 If the response uses an encoding, then it is specified in this header
16352 field.
16353 Transfer-Encoding: chunked means that the response is sent in parts,
16355 a method known as chunked transfer encoding. This is used when
16356 CouchDB does not know beforehand the size of the data it will send
16357 (for example, the changes feed).
16358 JSON Basics
16360 The majority of requests and responses to CouchDB use the JavaScript
16361 Object Notation (JSON) for formatting the content and structure of the
16362 data and responses.
16363 JSON is used because it is the simplest and easiest solution for work‐
16365 ing with data within a web browser, as JSON structures can be evaluated
16366 and used as JavaScript objects within the web browser environment. JSON
16367 also integrates with the server-side JavaScript used within CouchDB.
16368 JSON supports the same basic types as supported by JavaScript, these
16370 are:
16371 · Array - a list of values enclosed in square brackets. For example:
16373 ["one", "two", "three"]
16375 · Boolean - a true or false value. You can use these strings directly.
16377 For example:
16378 { "value": true}
16380 · Number - an integer or floating-point number.
16382 · Object - a set of key/value pairs (i.e. an associative array, or
16384 hash). The key must be a string, but the value can be any of the sup‐
16385 ported JSON values. For example:
16386 {
16388 "servings" : 4,
16389 "subtitle" : "Easy to make in advance, and then cook when ready",
16390 "cooktime" : 60,
16391 "title" : "Chicken Coriander"
16392 }
16393 In CouchDB, the JSON object is used to represent a variety of struc‐
16395 tures, including the main CouchDB document.
16396 · String - this should be enclosed by double-quotes and supports Uni‐
16398 code characters and backslash escaping. For example:
16399 "A String"
16401 Parsing JSON into a JavaScript object is supported through the
16403 JSON.parse() function in JavaScript, or through various libraries that
16404 will perform the parsing of the content into a JavaScript object for
16405 you. Libraries for parsing and generating JSON are available in many
16406 languages, including Perl, Python, Ruby, Erlang and others.
16407 WARNING:
16409 Care should be taken to ensure that your JSON structures are valid,
16410 invalid structures will cause CouchDB to return an HTTP status code
16411 of 500 (server error).
16412 Number Handling
16414 Developers and users new to computer handling of numbers often
16415 encounter surprises when expecting that a number stored in JSON format
16416 does not necessarily return as the same number as compared character by
16417 character.
16418 Any numbers defined in JSON that contain a decimal point or exponent
16420 will be passed through the Erlang VM’s idea of the “double” data type.
16421 Any numbers that are used in views will pass through the view server’s
16422 idea of a number (the common JavaScript case means even integers pass
16423 through a double due to JavaScript’s definition of a number).
16424 Consider this document that we write to CouchDB:
16426 {
16428 "_id":"30b3b38cdbd9e3a587de9b8122000cff",
16429 "number": 1.1
16430 }
16431 Now let’s read that document back from CouchDB:
16433 {
16435 "_id":"30b3b38cdbd9e3a587de9b8122000cff",
16436 "_rev":"1-f065cee7c3fd93aa50f6c97acde93030",
16437 "number":1.1000000000000000888
16438 }
16439 What happens is CouchDB is changing the textual representation of the
16441 result of decoding what it was given into some numerical format. In
16442 most cases this is an IEEE 754 double precision floating point number
16443 which is exactly what almost all other languages use as well.
16444 What Erlang does a bit differently than other languages is that it does
16446 not attempt to pretty print the resulting output to use the shortest
16447 number of characters. For instance, this is why we have this relation‐
16448 ship:
16449 ejson:encode(ejson:decode(<<"1.1">>)).
16451 <<"1.1000000000000000888">>
16452 What can be confusing here is that internally those two formats decode
16454 into the same IEEE-754 representation. And more importantly, it will
16455 decode into a fairly close representation when passed through all major
16456 parsers that we know about.
16457 While we’ve only been discussing cases where the textual representation
16459 changes, another important case is when an input value contains more
16460 precision than can actually represented in a double. (You could argue
16461 that this case is actually “losing” data if you don’t accept that num‐
16462 bers are stored in doubles).
16463 Here’s a log for a couple of the more common JSON libraries that happen
16465 to be on the author’s machine:
16466 Ejson (CouchDB’s current parser) at CouchDB sha 168a663b:
16468 $ ./utils/run -i
16470 Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2]
16471 [async-threads:4] [hipe] [kernel-poll:true]
16472 Eshell V5.8.5 (abort with ^G)
16474 1> ejson:encode(ejson:decode(<<"1.01234567890123456789012345678901234567890">>)).
16475 <<"1.0123456789012346135">>
16476 2> F = ejson:encode(ejson:decode(<<"1.01234567890123456789012345678901234567890">>)).
16477 <<"1.0123456789012346135">>
16478 3> ejson:encode(ejson:decode(F)).
16479 <<"1.0123456789012346135">>
16480 Node:
16482 $ node -v
16484 v0.6.15
16485 $ node
16486 JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16487 '1.0123456789012346'
16488 var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16489 undefined
16490 JSON.stringify(JSON.parse(f))
16491 '1.0123456789012346'
16492 Python:
16494 $ python
16496 Python 2.7.2 (default, Jun 20 2012, 16:23:33)
16497 [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
16498 Type "help", "copyright", "credits" or "license" for more information.
16499 import json
16500 json.dumps(json.loads("1.01234567890123456789012345678901234567890"))
16501 '1.0123456789012346'
16502 f = json.dumps(json.loads("1.01234567890123456789012345678901234567890"))
16503 json.dumps(json.loads(f))
16504 '1.0123456789012346'
16505 Ruby:
16507 $ irb --version
16509 irb 0.9.5(05/04/13)
16510 require 'JSON'
16511 => true
16512 JSON.dump(JSON.load("[1.01234567890123456789012345678901234567890]"))
16513 => "[1.01234567890123]"
16514 f = JSON.dump(JSON.load("[1.01234567890123456789012345678901234567890]"))
16515 => "[1.01234567890123]"
16516 JSON.dump(JSON.load(f))
16517 => "[1.01234567890123]"
16518 NOTE:
16520 A small aside on Ruby, it requires a top level object or array, so I
16521 just wrapped the value. Should be obvious it doesn’t affect the
16522 result of parsing the number though.
16523 Spidermonkey:
16525 $ js -h 2>&1 | head -n 1
16527 JavaScript-C 1.8.5 2011-03-31
16528 $ js
16529 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16530 "1.0123456789012346"
16531 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
16532 js> JSON.stringify(JSON.parse(f))
16533 "1.0123456789012346"
16534 As you can see they all pretty much behave the same except for Ruby
16536 actually does appear to be losing some precision over the other
16537 libraries.
16538 The astute observer will notice that ejson (the CouchDB JSON library)
16540 reported an extra three digits. While its tempting to think that this
16541 is due to some internal difference, its just a more specific case of
16542 the 1.1 input as described above.
16543 The important point to realize here is that a double can only hold a
16545 finite number of values. What we’re doing here is generating a string
16546 that when passed through the “standard” floating point parsing algo‐
16547 rithms (ie, strtod) will result in the same bit pattern in memory as we
16548 started with. Or, slightly different, the bytes in a JSON serialized
16549 number are chosen such that they refer to a single specific value that
16550 a double can represent.
16551 The important point to understand is that we’re mapping from one infi‐
16553 nite set onto a finite set. An easy way to see this is by reflecting on
16554 this:
16555 1.0 == 1.00 == 1.000 = 1.(infinite zeros)
16557 Obviously a computer can’t hold infinite bytes so we have to decimate
16559 our infinitely sized set to a finite set that can be represented con‐
16560 cisely.
16561 The game that other JSON libraries are playing is merely:
16563 “How few characters do I have to use to select this specific value for
16565 a double”
16566 And that game has lots and lots of subtle details that are difficult to
16568 duplicate in C without a significant amount of effort (it took Python
16569 over a year to get it sorted with their fancy build systems that auto‐
16570 matically run on a number of different architectures).
16571 Hopefully we’ve shown that CouchDB is not doing anything “funky” by
16573 changing input. Its behaving the same as any other common JSON library
16574 does, its just not pretty printing its output.
16575 On the other hand, if you actually are in a position where an IEEE-754
16577 double is not a satisfactory data type for your numbers, then the
16578 answer as has been stated is to not pass your numbers through this rep‐
16579 resentation. In JSON this is accomplished by encoding them as a string
16580 or by using integer types (although integer types can still bite you if
16581 you use a platform that has a different integer representation than
16582 normal, ie, JavaScript).
16583 Further information can be found easily, including the Floating Point
16585 Guide, and David Goldberg’s Reference.
16586 Also, if anyone is really interested in changing this behavior, we’re
16588 all ears for contributions to jiffy (which is theoretically going to
16589 replace ejson when we get around to updating the build system). The
16590 places we’ve looked for inspiration are TCL and Python. If you know a
16591 decent implementation of this float printing algorithm give us a
16592 holler.
16593 HTTP Status Codes
16595 With the interface to CouchDB working through HTTP, error codes and
16596 statuses are reported using a combination of the HTTP status code num‐
16597 ber, and corresponding data in the body of the response data.
16598 A list of the error codes returned by CouchDB, and generic descriptions
16600 of the related errors are provided below. The meaning of different sta‐
16601 tus codes for specific request types are provided in the corresponding
16602 API call reference.
16603 · 200 - OK
16605 Request completed successfully.
16607 · 201 - Created
16609 Document created successfully.
16611 · 202 - Accepted
16613 Request has been accepted, but the corresponding operation may not
16615 have completed. This is used for background operations, such as data‐
16616 base compaction.
16617 · 304 - Not Modified
16619 The additional content requested has not been modified. This is used
16621 with the ETag system to identify the version of information returned.
16622 · 400 - Bad Request
16624 Bad request structure. The error can indicate an error with the
16626 request URL, path or headers. Differences in the supplied MD5 hash
16627 and content also trigger this error, as this may indicate message
16628 corruption.
16629 · 401 - Unauthorized
16631 The item requested was not available using the supplied authoriza‐
16633 tion, or authorization was not supplied.
16634 · 403 - Forbidden
16636 The requested item or operation is forbidden.
16638 · 404 - Not Found
16640 The requested content could not be found. The content will include
16642 further information, as a JSON object, if available. The structure
16643 will contain two keys, error and reason. For example:
16644 {"error":"not_found","reason":"no_db_file"}
16646 · 405 - Method Not Allowed
16648 A request was made using an invalid HTTP request type for the URL
16650 requested. For example, you have requested a PUT when a POST is
16651 required. Errors of this type can also triggered by invalid URL
16652 strings.
16653 · 406 - Not Acceptable
16655 The requested content type is not supported by the server.
16657 · 409 - Conflict
16659 Request resulted in an update conflict.
16661 · 412 - Precondition Failed
16663 The request headers from the client and the capabilities of the
16665 server do not match.
16666 · 413 - Request Entity Too Large
16668 A document exceeds the configured couchdb/max_document_size value or
16670 the entire request exceeds the httpd/max_http_request_size value.
16671 · 415 - Unsupported Media Type
16673 The content types supported, and the content type of the information
16675 being requested or submitted indicate that the content type is not
16676 supported.
16677 · 416 - Requested Range Not Satisfiable
16679 The range specified in the request header cannot be satisfied by the
16681 server.
16682 · 417 - Expectation Failed
16684 When sending documents in bulk, the bulk load operation failed.
16686 · 500 - Internal Server Error
16688 The request was invalid, either because the supplied JSON was
16690 invalid, or invalid information was supplied as part of the request.
16691 Server
16693 The CouchDB server interface provides the basic interface to a CouchDB
16694 server for obtaining CouchDB information and getting and setting con‐
16695 figuration information.
16696 /
16698 GET / Accessing the root of a CouchDB instance returns meta informa‐
16699 tion about the instance. The response is a JSON structure con‐
16700 taining information about the server, including a welcome mes‐
16701 sage and the version of the server.
16702 Request Headers
16704 · Accept – .INDENT 2.0
16706 · application/json
16708 · text/plain
16710 Response Headers
16713 · Content-Type – .INDENT 2.0
16715 · application/json
16717 · text/plain; charset=utf-8
16719 Status Codes
16722 · 200 OK – Request completed successfully
16724 Request:
16726 GET / HTTP/1.1
16728 Accept: application/json
16729 Host: localhost:5984
16730 Response:
16732 HTTP/1.1 200 OK
16734 Cache-Control: must-revalidate
16735 Content-Length: 179
16736 Content-Type: application/json
16737 Date: Sat, 10 Aug 2013 06:33:33 GMT
16738 Server: CouchDB (Erlang/OTP)
16739 {
16741 "couchdb": "Welcome",
16742 "uuid": "85fb71bf700c17267fef77535820e371",
16743 "vendor": {
16744 "name": "The Apache Software Foundation",
16745 "version": "1.3.1"
16746 },
16747 "version": "1.3.1"
16748 }
16749 /_active_tasks
16751 Changed in version 2.1.0: Because of how the scheduling replicator
16752 works, continuous replication jobs could be periodically stopped and
16753 then started later. When they are not running they will not appear in
16754 the _active_tasks endpoint
16755 GET /_active_tasks
16758 List of running tasks, including the task type, name, status and
16759 process ID. The result is a JSON array of the currently running
16760 tasks, with each task being described with a single object.
16761 Depending on operation type set of response object fields might
16762 be different.
16763 Request Headers
16765 · Accept – .INDENT 2.0
16767 · application/json
16769 · text/plain
16771 Response Headers
16774 · Content-Type – .INDENT 2.0
16776 · application/json
16778 · text/plain; charset=utf-8
16780 Response JSON Object
16783 · changes_done (number) – Processed changes
16785 · database (string) – Source database
16787 · pid (string) – Process ID
16789 · progress (number) – Current percentage progress
16791 · started_on (number) – Task start time as unix timestamp
16793 · status (string) – Task status message
16795 · task (string) – Task name
16797 · total_changes (number) – Total changes to process
16799 · type (string) – Operation Type
16801 · updated_on (number) – Unix timestamp of last operation update
16803 Status Codes
16805 · 200 OK – Request completed successfully
16807 · 401 Unauthorized – CouchDB Server Administrator privileges
16809 required
16810 Request:
16812 GET /_active_tasks HTTP/1.1
16814 Accept: application/json
16815 Host: localhost:5984
16816 Response:
16818 HTTP/1.1 200 OK
16820 Cache-Control: must-revalidate
16821 Content-Length: 1690
16822 Content-Type: application/json
16823 Date: Sat, 10 Aug 2013 06:37:31 GMT
16824 Server: CouchDB (Erlang/OTP)
16825 [
16827 {
16828 "changes_done": 64438,
16829 "database": "mailbox",
16830 "pid": "<0.12986.1>",
16831 "progress": 84,
16832 "started_on": 1376116576,
16833 "total_changes": 76215,
16834 "type": "database_compaction",
16835 "updated_on": 1376116619
16836 },
16837 {
16838 "changes_done": 14443,
16839 "database": "mailbox",
16840 "design_document": "c9753817b3ba7c674d92361f24f59b9f",
16841 "pid": "<0.10461.3>",
16842 "progress": 18,
16843 "started_on": 1376116621,
16844 "total_changes": 76215,
16845 "type": "indexer",
16846 "updated_on": 1376116650
16847 },
16848 {
16849 "changes_done": 5454,
16850 "database": "mailbox",
16851 "design_document": "_design/meta",
16852 "pid": "<0.6838.4>",
16853 "progress": 7,
16854 "started_on": 1376116632,
16855 "total_changes": 76215,
16856 "type": "indexer",
16857 "updated_on": 1376116651
16858 },
16859 {
16860 "checkpointed_source_seq": 68585,
16861 "continuous": false,
16862 "doc_id": null,
16863 "doc_write_failures": 0,
16864 "docs_read": 4524,
16865 "docs_written": 4524,
16866 "missing_revisions_found": 4524,
16867 "pid": "<0.1538.5>",
16868 "progress": 44,
16869 "replication_id": "9bc1727d74d49d9e157e260bb8bbd1d5",
16870 "revisions_checked": 4524,
16871 "source": "mailbox",
16872 "source_seq": 154419,
16873 "started_on": 1376116644,
16874 "target": "http://mailsrv:5984/mailbox",
16875 "type": "replication",
16876 "updated_on": 1376116651
16877 }
16878 ]
16879 /_all_dbs
16881 GET /_all_dbs
16882 Returns a list of all the databases in the CouchDB instance.
16883 Request Headers
16885 · Accept – .INDENT 2.0
16887 · application/json
16889 · text/plain
16891 Query Parameters
16894 · descending (boolean) – Return the databases in descending
16896 order by key. Default is false.
16897 · endkey (json) – Stop returning databases when the specified
16899 key is reached.
16900 · end_key (json) – Alias for endkey param
16902 · limit (number) – Limit the number of the returned databases to
16904 the specified number.
16905 · skip (number) – Skip this number of databases before starting
16907 to return the results. Default is 0.
16908 · startkey (json) – Return databases starting with the specified
16910 key.
16911 · start_key (json) – Alias for startkey.
16913 Response Headers
16915 · Content-Type – .INDENT 2.0
16917 · application/json
16919 · text/plain; charset=utf-8
16921 Status Codes
16924 · 200 OK – Request completed successfully
16926 Request:
16928 GET /_all_dbs HTTP/1.1
16930 Accept: application/json
16931 Host: localhost:5984
16932 Response:
16934 HTTP/1.1 200 OK
16936 Cache-Control: must-revalidate
16937 Content-Length: 52
16938 Content-Type: application/json
16939 Date: Sat, 10 Aug 2013 06:57:48 GMT
16940 Server: CouchDB (Erlang/OTP)
16941 [
16943 "_users",
16944 "contacts",
16945 "docs",
16946 "invoices",
16947 "locations"
16948 ]
16949 /_dbs_info
16951 New in version 2.2.
16952 POST /_dbs_info
16955 Returns information of a list of the specified databases in the
16956 CouchDB instance. This enables you to request information about
16957 multiple databases in a single request, in place of multiple GET
16958 /{db} requests.
16959 Request Headers
16961 · Accept – .INDENT 2.0
16963 · application/json
16965 Response Headers
16968 · Content-Type – .INDENT 2.0
16970 · application/json
16972 Request JSON Object
16975 · keys (array) – Array of database names to be requested
16977 Status Codes
16979 · 200 OK – Request completed successfully
16981 · 400 Bad Request – Missing keys or exceeded keys in request
16983 Request:
16985 POST /_dbs_info HTTP/1.1
16987 Accept: application/json
16988 Host: localhost:5984
16989 Content-Type: application/json
16990 {
16992 "keys": [
16993 "animals",
16994 "plants"
16995 ]
16996 }
16997 Response:
16999 HTTP/1.1 200 OK
17001 Cache-Control: must-revalidate
17002 Content-Type: application/json
17003 Date: Sat, 20 Dec 2017 06:57:48 GMT
17004 Server: CouchDB (Erlang/OTP)
17005 [
17007 {
17008 "key": "animals",
17009 "info": {
17010 "db_name": "animals",
17011 "update_seq": "52232",
17012 "sizes": {
17013 "file": 1178613587,
17014 "external": 1713103872,
17015 "active": 1162451555
17016 },
17017 "purge_seq": 0,
17018 "doc_del_count": 0,
17019 "doc_count": 52224,
17020 "disk_format_version": 6,
17021 "compact_running": false,
17022 "cluster": {
17023 "q": 8,
17024 "n": 3,
17025 "w": 2,
17026 "r": 2
17027 },
17028 "instance_start_time": "0"
17029 }
17030 },
17031 {
17032 "key": "plants",
17033 "info": {
17034 "db_name": "plants",
17035 "update_seq": "303",
17036 "sizes": {
17037 "file": 3872387,
17038 "external": 2339,
17039 "active": 67475
17040 },
17041 "purge_seq": 0,
17042 "doc_del_count": 0,
17043 "doc_count": 11,
17044 "disk_format_version": 6,
17045 "compact_running": false,
17046 "cluster": {
17047 "q": 8,
17048 "n": 3,
17049 "w": 2,
17050 "r": 2
17051 },
17052 "instance_start_time": "0"
17053 }
17054 }
17055 ]
17056 NOTE:
17058 The supported number of the specified databases in the list can be
17059 limited by modifying the max_db_number_for_dbs_info_req entry in
17060 configuration file. The default limit is 100.
17061 /_cluster_setup
17063 New in version 2.0.
17064 GET /_cluster_setup
17067 Returns the status of the node or cluster, per the cluster setup
17068 wizard.
17069 Request Headers
17071 · Accept – .INDENT 2.0
17073 · application/json
17075 · text/plain
17077 Query Parameters
17080 · ensure_dbs_exist (array) – List of system databases to ensure
17082 exist on the node/cluster. Defaults to ["_users","_replica‐
17083 tor","_global_changes"].
17084 Response Headers
17086 · Content-Type – .INDENT 2.0
17088 · application/json
17090 · text/plain; charset=utf-8
17092 Response JSON Object
17095 · state (string) – Current state of the node and/or cluster (see
17097 below)
17098 Status Codes
17100 · 200 OK – Request completed successfully
17102 The state returned indicates the current node or cluster state, and is
17104 one of the following:
17105 · cluster_disabled: The current node is completely unconfigured.
17107 · single_node_disabled: The current node is configured as a sin‐
17109 gle (standalone) node ([cluster] n=1), but either does not
17110 have a server-level admin user defined, or does not have the
17111 standard system databases created. If the ensure_dbs_exist
17112 query parameter is specified, the list of databases provided
17113 overrides the default list of standard system databases.
17114 · single_node_enabled: The current node is configured as a sin‐
17116 gle (standalone) node, has a server-level admin user defined,
17117 and has the ensure_dbs_exist list (explicit or default) of
17118 databases created.
17119 · cluster_enabled: The current node has [cluster] n > 1, is not
17121 bound to 127.0.0.1 and has a server-level admin user defined.
17122 However, the full set of standard system databases have not
17123 been created yet. If the ensure_dbs_exist query parameter is
17124 specified, the list of databases provided overrides the
17125 default list of standard system databases.
17126 · cluster_finished: The current node has [cluster] n > 1, is not
17128 bound to 127.0.0.1, has a server-level admin user defined and
17129 has the ensure_dbs_exist list (explicit or default) of data‐
17130 bases created.
17131 Request:
17133 GET /_cluster_setup HTTP/1.1
17135 Accept: application/json
17136 Host: localhost:5984
17137 Response:
17139 HTTP/1.1 200 OK
17141 X-CouchDB-Body-Time: 0
17142 X-Couch-Request-ID: 5c058bdd37
17143 Server: CouchDB/2.1.0-7f17678 (Erlang OTP/17)
17144 Date: Sun, 30 Jul 2017 06:33:18 GMT
17145 Content-Type: application/json
17146 Content-Length: 29
17147 Cache-Control: must-revalidate
17148 {"state":"cluster_enabled"}
17150 POST /_cluster_setup
17152 Configure a node as a single (standalone) node, as part of a
17153 cluster, or finalise a cluster.
17154 Request Headers
17156 · Accept – .INDENT 2.0
17158 · application/json
17160 · text/plain
17162 · Content-Type – application/json
17165 Request JSON Object
17167 · action (string) – .INDENT 2.0
17169 · enable_single_node: Configure the current node as a single,
17171 standalone CouchDB server.
17172 · enable_cluster: Configure the local or remote node as one
17174 node, preparing it to be joined to a new CouchDB cluster.
17175 · add_node: Add the specified remote node to this cluster’s list
17177 of nodes, joining it to the cluster.
17178 · finish_cluster: Finalise the cluster by creating the standard
17180 system databases.
17181 · bind_address (string) – The IP address to which to bind the current
17184 node. The special value 0.0.0.0 may be specified to bind to all
17185 interfaces on the host. (enable_cluster and enable_single_node only)
17186 · username (string) – The username of the server-level administrator to
17188 create. (enable_cluster and enable_single_node only), or the remote
17189 server’s administrator username (add_node)
17190 · password (string) – The password for the server-level administrator
17192 to create. (enable_cluster and enable_single_node only), or the
17193 remote server’s administrator username (add_node)
17194 · port (number) – The TCP port to which to bind this node (enable_clus‐
17196 ter and enable_single_node only) or the TCP port to which to bind a
17197 remote node (add_node only).
17198 · node_count (number) – The total number of nodes to be joined into the
17200 cluster, including this one. Used to determine the value of the clus‐
17201 ter’s n, up to a maximum of 3. (enable_cluster only)
17202 · remote_node (string) – The IP address of the remote node to setup as
17204 part of this cluster’s list of nodes. (enable_cluster only)
17205 · remote_current_user (string) – The username of the server-level
17207 administrator authorized on the remote node. (enable_cluster only)
17208 · remote_current_password (string) – The password of the server-level
17210 administrator authorized on the remote node. (enable_cluster only)
17211 · host (string) – The remote node IP of the node to add to the cluster.
17213 (add_node only)
17214 · ensure_dbs_exist (array) – List of system databases to ensure exist
17216 on the node/cluster. Defaults to ["_users","_replica‐
17217 tor","_global_changes"].
17218 No example request/response included here. For a worked example, please
17220 see cluster/setup/api.
17221 /_db_updates
17223 New in version 1.4.
17224 GET /_db_updates
17227 Returns a list of all database events in the CouchDB instance.
17228 Request Headers
17230 · Accept – .INDENT 2.0
17232 · application/json
17234 · text/plain
17236 Query Parameters
17239 · feed (string) – .INDENT 2.0
17241 · normal: Returns all historical DB changes, then closes the
17243 connection. Default.
17244 · longpoll: Closes the connection after the first event.
17246 · continuous: Send a line of JSON per event. Keeps the socket
17248 open until timeout.
17249 · eventsource: Like, continuous, but sends the events in
17251 EventSource format.
17252 · timeout (number) – Number of seconds until CouchDB closes the connec‐
17255 tion. Default is 60.
17256 · heartbeat (number) – Period in milliseconds after which an empty line
17258 is sent in the results. Only applicable for longpoll, continuous, and
17259 eventsource feeds. Overrides any timeout to keep the feed alive
17260 indefinitely. Default is 60000. May be true to use default value.
17261 · since (string) – Return only updates since the specified sequence ID.
17263 May be the string now to begin showing only new updates.
17264 Response Headers
17266 · Content-Type – .INDENT 2.0
17268 · application/json
17270 · text/plain; charset=utf-8
17272 · Transfer-Encoding – chunked
17275 Response JSON Object
17277 · results (array) – An array of database events. For longpoll
17279 and continuous modes, the entire response is the contents of
17280 the results array.
17281 · last_seq (string) – The last sequence ID reported.
17283 Status Codes
17285 · 200 OK – Request completed successfully
17287 · 401 Unauthorized – CouchDB Server Administrator privileges
17289 required
17290 The results field of database updates:
17292 JSON Object
17294 · db_name (string) – Database name.
17296 · type (string) – A database event is one of created,
17298 updated, deleted.
17299 · seq (json) – Update sequence of the event.
17301 Request:
17303 GET /_db_updates HTTP/1.1
17305 Accept: application/json
17306 Host: localhost:5984
17307 Response:
17309 HTTP/1.1 200 OK
17311 Cache-Control: must-revalidate
17312 Content-Type: application/json
17313 Date: Sat, 18 Mar 2017 19:01:35 GMT
17314 Etag: "C1KU98Y6H0LGM7EQQYL6VSL07"
17315 Server: CouchDB/2.0.0 (Erlang OTP/17)
17316 Transfer-Encoding: chunked
17317 X-Couch-Request-ID: ad87efc7ff
17318 X-CouchDB-Body-Time: 0
17319 {
17321 "results":[
17322 {"db_name":"mailbox","type":"created","seq":"1-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZExFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4h"},
17323 {"db_name":"mailbox","type":"deleted","seq":"2-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZEpFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4hdQsg6vYTUncAou4-IXUPIOpA7ssCAIFHa60"},
17324 ],
17325 "last_seq": "2-g1AAAAFReJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOZEpFyjAnmJhkWaeaIquGIf2JAUgmWQPMiGRAZcaB5CaePxqEkBq6vGqyWMBkgwNQAqobD4hdQsg6vYTUncAou4-IXUPIOpA7ssCAIFHa60"
17326 }
17327 /_membership
17329 New in version 2.0.
17330 GET /_membership
17333 Displays the nodes that are part of the cluster as clus‐
17334 ter_nodes. The field all_nodes displays all nodes this node
17335 knows about, including the ones that are part of the cluster.
17336 The endpoint is useful when setting up a cluster, see clus‐
17337 ter/nodes
17338 Request Headers
17340 · Accept – .INDENT 2.0
17342 · application/json
17344 · text/plain
17346 Response Headers
17349 · Content-Type – .INDENT 2.0
17351 · application/json
17353 · text/plain; charset=utf-8
17355 Status Codes
17358 · 200 OK – Request completed successfully
17360 Request:
17362 GET /_membership HTTP/1.1
17364 Accept: application/json
17365 Host: localhost:5984
17366 Response:
17368 HTTP/1.1 200 OK
17370 Cache-Control: must-revalidate
17371 Content-Type: application/json
17372 Date: Sat, 11 Jul 2015 07:02:41 GMT
17373 Server: CouchDB (Erlang/OTP)
17374 Content-Length: 142
17375 {
17377 "all_nodes": [
17378 "node1@127.0.0.1",
17379 "node2@127.0.0.1",
17380 "node3@127.0.0.1"
17381 ],
17382 "cluster_nodes": [
17383 "node1@127.0.0.1",
17384 "node2@127.0.0.1",
17385 "node3@127.0.0.1"
17386 ]
17387 }
17388 /_replicate
17390 POST /_replicate
17391 Request, configure, or stop, a replication operation.
17392 Request Headers
17394 · Accept – .INDENT 2.0
17396 · application/json
17398 · text/plain
17400 · Content-Type – application/json
17403 Request JSON Object
17405 · cancel (boolean) – Cancels the replication
17407 · continuous (boolean) – Configure the replication to be contin‐
17409 uous
17410 · create_target (boolean) – Creates the target database.
17412 Required administrator’s privileges on target server.
17413 · doc_ids (array) – Array of document IDs to be synchronized
17415 · filter (string) – The name of a filter function.
17417 · source_proxy (string) – Address of a proxy server through
17419 which replication from the source should occur (protocol can
17420 be “http” or “socks5”)
17421 · target_proxy (string) – Address of a proxy server through
17423 which replication to the target should occur (protocol can be
17424 “http” or “socks5”)
17425 · source (string/object) – Fully qualified source database URL
17427 or an object which contains the full URL of the source data‐
17428 base with additional parameters like headers. Eg: ‘‐
17429 http://example.com/source_db_name’ or {“url”:”url in here”,
17430 “headers”: {“header1”:”value1”, …}} . For backwards compati‐
17431 bility, CouchDB 3.x will auto-convert bare database names by
17432 prepending the address and port CouchDB is listening on, to
17433 form a complete URL. This behaviour is deprecated in 3.x and
17434 will be removed in CouchDB 4.0.
17435 · target (string/object) – Fully qualified target database URL
17437 or an object which contains the full URL of the target data‐
17438 base with additional parameters like headers. Eg: ‘‐
17439 http://example.com/target_db_name’ or {“url”:”url in here”,
17440 “headers”: {“header1”:”value1”, …}} . For backwards compati‐
17441 bility, CouchDB 3.x will auto-convert bare database names by
17442 prepending the address and port CouchDB is listening on, to
17443 form a complete URL. This behaviour is deprecated in 3.x and
17444 will be removed in CouchDB 4.0.
17445 Response Headers
17447 · Content-Type – .INDENT 2.0
17449 · application/json
17451 · text/plain; charset=utf-8
17453 Response JSON Object
17456 · history (array) – Replication history (see below)
17458 · ok (boolean) – Replication status
17460 · replication_id_version (number) – Replication protocol version
17462 · session_id (string) – Unique session ID
17464 · source_last_seq (number) – Last sequence number read from
17466 source database
17467 Status Codes
17469 · 200 OK – Replication request successfully completed
17471 · 202 Accepted – Continuous replication request has been
17473 accepted
17474 · 400 Bad Request – Invalid JSON data
17476 · 401 Unauthorized – CouchDB Server Administrator privileges
17478 required
17479 · 404 Not Found – Either the source or target DB is not found or
17481 attempt to cancel unknown replication task
17482 · 500 Internal Server Error – JSON specification was invalid
17484The specification of the replication request is controlled through the JSON
17486content of the request. The JSON should be an object with the fields defining
17487the source, target and other options.
17488The Replication history is an array of objects with following structure:
17490 JSON Object
17492 · doc_write_failures (number) – Number of document write
17494 failures
17495 · docs_read (number) – Number of documents read
17497 · docs_written (number) – Number of documents written to
17499 target
17500 · end_last_seq (number) – Last sequence number in changes
17502 stream
17503 · end_time (string) – Date/Time replication operation
17505 completed in RFC 2822 format
17506 · missing_checked (number) – Number of missing documents
17508 checked
17509 · missing_found (number) – Number of missing documents
17511 found
17512 · recorded_seq (number) – Last recorded sequence number
17514 · session_id (string) – Session ID for this replication
17516 operation
17517 · start_last_seq (number) – First sequence number in
17519 changes stream
17520 · start_time (string) – Date/Time replication operation
17522 started in RFC 2822 format
17523
17525 As of CouchDB 2.0.0, fully qualified URLs are required for both the
17526 replication source and target parameters.
17527 Request
17529 POST /_replicate HTTP/1.1
17531 Accept: application/json
17532 Content-Length: 80
17533 Content-Type: application/json
17534 Host: localhost:5984
17535 {
17537 "source": "http://127.0.0.1:5984/db_a",
17538 "target": "http://127.0.0.1:5984/db_b"
17539 }
17540 Response
17542 HTTP/1.1 200 OK
17544 Cache-Control: must-revalidate
17545 Content-Length: 692
17546 Content-Type: application/json
17547 Date: Sun, 11 Aug 2013 20:38:50 GMT
17548 Server: CouchDB (Erlang/OTP)
17549 {
17551 "history": [
17552 {
17553 "doc_write_failures": 0,
17554 "docs_read": 10,
17555 "docs_written": 10,
17556 "end_last_seq": 28,
17557 "end_time": "Sun, 11 Aug 2013 20:38:50 GMT",
17558 "missing_checked": 10,
17559 "missing_found": 10,
17560 "recorded_seq": 28,
17561 "session_id": "142a35854a08e205c47174d91b1f9628",
17562 "start_last_seq": 1,
17563 "start_time": "Sun, 11 Aug 2013 20:38:50 GMT"
17564 },
17565 {
17566 "doc_write_failures": 0,
17567 "docs_read": 1,
17568 "docs_written": 1,
17569 "end_last_seq": 1,
17570 "end_time": "Sat, 10 Aug 2013 15:41:54 GMT",
17571 "missing_checked": 1,
17572 "missing_found": 1,
17573 "recorded_seq": 1,
17574 "session_id": "6314f35c51de3ac408af79d6ee0c1a09",
17575 "start_last_seq": 0,
17576 "start_time": "Sat, 10 Aug 2013 15:41:54 GMT"
17577 }
17578 ],
17579 "ok": true,
17580 "replication_id_version": 3,
17581 "session_id": "142a35854a08e205c47174d91b1f9628",
17582 "source_last_seq": 28
17583 }
17584 Replication Operation
17586 The aim of the replication is that at the end of the process, all
17587 active documents on the source database are also in the destination
17588 database and all documents that were deleted in the source databases
17589 are also deleted (if they exist) on the destination database.
17590 Replication can be described as either push or pull replication:
17592 · Pull replication is where the source is the remote CouchDB instance,
17594 and the target is the local database.
17595 Pull replication is the most useful solution to use if your source
17597 database has a permanent IP address, and your destination (local)
17598 database may have a dynamically assigned IP address (for example,
17599 through DHCP). This is particularly important if you are replicating
17600 to a mobile or other device from a central server.
17601 · Push replication is where the source is a local database, and target
17603 is a remote database.
17604 Specifying the Source and Target Database
17606 You must use the URL specification of the CouchDB database if you want
17607 to perform replication in either of the following two situations:
17608 · Replication with a remote database (i.e. another instance of CouchDB
17610 on the same host, or a different host)
17611 · Replication with a database that requires authentication
17613 For example, to request replication between a database local to the
17615 CouchDB instance to which you send the request, and a remote database
17616 you might use the following request:
17617 POST http://couchdb:5984/_replicate HTTP/1.1
17619 Content-Type: application/json
17620 Accept: application/json
17621 {
17623 "source" : "recipes",
17624 "target" : "http://coucdb-remote:5984/recipes",
17625 }
17626 In all cases, the requested databases in the source and target specifi‐
17628 cation must exist. If they do not, an error will be returned within the
17629 JSON object:
17630 {
17632 "error" : "db_not_found"
17633 "reason" : "could not open http://couchdb-remote:5984/ol1ka/",
17634 }
17635 You can create the target database (providing your user credentials
17637 allow it) by adding the create_target field to the request object:
17638 POST http://couchdb:5984/_replicate HTTP/1.1
17640 Content-Type: application/json
17641 Accept: application/json
17642 {
17644 "create_target" : true
17645 "source" : "recipes",
17646 "target" : "http://couchdb-remote:5984/recipes",
17647 }
17648 The create_target field is not destructive. If the database already
17650 exists, the replication proceeds as normal.
17651 Single Replication
17653 You can request replication of a database so that the two databases can
17654 be synchronized. By default, the replication process occurs one time
17655 and synchronizes the two databases together. For example, you can
17656 request a single synchronization between two databases by supplying the
17657 source and target fields within the request JSON content.
17658 POST http://couchdb:5984/_replicate HTTP/1.1
17660 Accept: application/json
17661 Content-Type: application/json
17662 {
17664 "source" : "recipes",
17665 "target" : "recipes-snapshot",
17666 }
17667 In the above example, the databases recipes and recipes-snapshot will
17669 be synchronized. These databases are local to the CouchDB instance
17670 where the request was made. The response will be a JSON structure con‐
17671 taining the success (or failure) of the synchronization process, and
17672 statistics about the process:
17673 {
17675 "ok" : true,
17676 "history" : [
17677 {
17678 "docs_read" : 1000,
17679 "session_id" : "52c2370f5027043d286daca4de247db0",
17680 "recorded_seq" : 1000,
17681 "end_last_seq" : 1000,
17682 "doc_write_failures" : 0,
17683 "start_time" : "Thu, 28 Oct 2010 10:24:13 GMT",
17684 "start_last_seq" : 0,
17685 "end_time" : "Thu, 28 Oct 2010 10:24:14 GMT",
17686 "missing_checked" : 0,
17687 "docs_written" : 1000,
17688 "missing_found" : 1000
17689 }
17690 ],
17691 "session_id" : "52c2370f5027043d286daca4de247db0",
17692 "source_last_seq" : 1000
17693 }
17694 Continuous Replication
17696 Synchronization of a database with the previously noted methods happens
17697 only once, at the time the replicate request is made. To have the tar‐
17698 get database permanently replicated from the source, you must set the
17699 continuous field of the JSON object within the request to true.
17700 With continuous replication changes in the source database are repli‐
17702 cated to the target database in perpetuity until you specifically
17703 request that replication ceases.
17704 POST http://couchdb:5984/_replicate HTTP/1.1
17706 Accept: application/json
17707 Content-Type: application/json
17708 {
17710 "continuous" : true
17711 "source" : "recipes",
17712 "target" : "http://couchdb-remote:5984/recipes",
17713 }
17714 Changes will be replicated between the two databases as long as a net‐
17716 work connection is available between the two instances.
17717 NOTE:
17719 Two keep two databases synchronized with each other, you need to set
17720 replication in both directions; that is, you must replicate from
17721 source to target, and separately from target to source.
17722 Canceling Continuous Replication
17724 You can cancel continuous replication by adding the cancel field to the
17725 JSON request object and setting the value to true. Note that the struc‐
17726 ture of the request must be identical to the original for the cancella‐
17727 tion request to be honoured. For example, if you requested continuous
17728 replication, the cancellation request must also contain the continuous
17729 field.
17730 For example, the replication request:
17732 POST http://couchdb:5984/_replicate HTTP/1.1
17734 Content-Type: application/json
17735 Accept: application/json
17736 {
17738 "source" : "recipes",
17739 "target" : "http://couchdb-remote:5984/recipes",
17740 "create_target" : true,
17741 "continuous" : true
17742 }
17743 Must be canceled using the request:
17745 POST http://couchdb:5984/_replicate HTTP/1.1
17747 Accept: application/json
17748 Content-Type: application/json
17749 {
17751 "cancel" : true,
17752 "continuous" : true
17753 "create_target" : true,
17754 "source" : "recipes",
17755 "target" : "http://couchdb-remote:5984/recipes",
17756 }
17757 Requesting cancellation of a replication that does not exist results in
17759 a 404 error.
17760 /_scheduler/jobs
17762 GET /_scheduler/jobs
17763 List of replication jobs. Includes replications created via
17764 /_replicate endpoint as well as those created from replication
17765 documents. Does not include replications which have completed or
17766 have failed to start because replication documents were mal‐
17767 formed. Each job description will include source and target
17768 information, replication id, a history of recent event, and a
17769 few other things.
17770 Request Headers
17772 · Accept – .INDENT 2.0
17774 · application/json
17776 Response Headers
17779 · Content-Type – .INDENT 2.0
17781 · application/json
17783 Query Parameters
17786 · limit (number) – How many results to return
17788 · skip (number) – How many result to skip starting at the begin‐
17790 ning, ordered by replication ID
17791 Response JSON Object
17793 · offset (number) – How many results were skipped
17795 · total_rows (number) – Total number of replication jobs
17797 · id (string) – Replication ID.
17799 · database (string) – Replication document database
17801 · doc_id (string) – Replication document ID
17803 · history (list) – Timestamped history of events as a list of
17805 objects
17806 · pid (string) – Replication process ID
17808 · node (string) – Cluster node where the job is running
17810 · source (string) – Replication source
17812 · target (string) – Replication target
17814 · start_time (string) – Timestamp of when the replication was
17816 started
17817 Status Codes
17819 · 200 OK – Request completed successfully
17821 · 401 Unauthorized – CouchDB Server Administrator privileges
17823 required
17824
17826 GET /_scheduler/jobs HTTP/1.1
17828 Accept: application/json
17829 Host: localhost:5984
17830 Response:
17832 HTTP/1.1 200 OK
17834 Cache-Control: must-revalidate
17835 Content-Length: 1690
17836 Content-Type: application/json
17837 Date: Sat, 29 Apr 2017 05:05:16 GMT
17838 Server: CouchDB (Erlang/OTP)
17839 {
17841 "jobs": [
17842 {
17843 "database": "_replicator",
17844 "doc_id": "cdyno-0000001-0000003",
17845 "history": [
17846 {
17847 "timestamp": "2017-04-29T05:01:37Z",
17848 "type": "started"
17849 },
17850 {
17851 "timestamp": "2017-04-29T05:01:37Z",
17852 "type": "added"
17853 }
17854 ],
17855 "id": "8f5b1bd0be6f9166ccfd36fc8be8fc22+continuous",
17856 "info": {
17857 "changes_pending": 0,
17858 "checkpointed_source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
17859 "doc_write_failures": 0,
17860 "docs_read": 113,
17861 "docs_written": 113,
17862 "missing_revisions_found": 113,
17863 "revisions_checked": 113,
17864 "source_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ",
17865 "through_seq": "113-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE01ygQLsZsYGqcamiZjKcRqRxwIkGRqA1H-oSbZgk1KMLCzTDE0wdWUBAF6HJIQ"
17866 },
17867 "node": "node1@127.0.0.1",
17868 "pid": "<0.1850.0>",
17869 "source": "http://myserver.com/foo",
17870 "start_time": "2017-04-29T05:01:37Z",
17871 "target": "http://adm:*****@localhost:15984/cdyno-0000003/",
17872 "user": null
17873 },
17874 {
17875 "database": "_replicator",
17876 "doc_id": "cdyno-0000001-0000002",
17877 "history": [
17878 {
17879 "timestamp": "2017-04-29T05:01:37Z",
17880 "type": "started"
17881 },
17882 {
17883 "timestamp": "2017-04-29T05:01:37Z",
17884 "type": "added"
17885 }
17886 ],
17887 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
17888 "info": {
17889 "changes_pending": null,
17890 "checkpointed_source_seq": 0,
17891 "doc_write_failures": 0,
17892 "docs_read": 12,
17893 "docs_written": 12,
17894 "missing_revisions_found": 12,
17895 "revisions_checked": 12,
17896 "source_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg",
17897 "through_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg"
17898 },
17899 "node": "node2@127.0.0.1",
17900 "pid": "<0.1757.0>",
17901 "source": "http://myserver.com/foo",
17902 "start_time": "2017-04-29T05:01:37Z",
17903 "target": "http://adm:*****@localhost:15984/cdyno-0000002/",
17904 "user": null
17905 }
17906 ],
17907 "offset": 0,
17908 "total_rows": 2
17909 }
17910 /_scheduler/docs
17912 Changed in version 2.1.0: Use this endpoint to monitor the state of
17913 document-based replications. Previously needed to poll both documents
17914 and _active_tasks to get a complete state summary
17915 Changed in version 3.0.0: In error states the “info” field switched
17918 from being a string to being an object
17919 GET /_scheduler/docs
17922 List of replication document states. Includes information about
17923 all the documents, even in completed and failed states. For each
17924 document it returns the document ID, the database, the replica‐
17925 tion ID, source and target, and other information.
17926 Request Headers
17928 · Accept – .INDENT 2.0
17930 · application/json
17932 Response Headers
17935 · Content-Type – .INDENT 2.0
17937 · application/json
17939 Query Parameters
17942 · limit (number) – How many results to return
17944 · skip (number) – How many result to skip starting at the begin‐
17946 ning, if ordered by document ID
17947 Response JSON Object
17949 · offset (number) – How many results were skipped
17951 · total_rows (number) – Total number of replication documents.
17953 · id (string) – Replication ID, or null if state is completed or
17955 failed
17956 · state (string) – One of following states (see replica‐
17958 tor/states for descriptions): initializing, running, com‐
17959 pleted, pending, crashing, error, failed
17960 · database (string) – Database where replication document came
17962 from
17963 · doc_id (string) – Replication document ID
17965 · node (string) – Cluster node where the job is running
17967 · source (string) – Replication source
17969 · target (string) – Replication target
17971 · start_time (string) – Timestamp of when the replication was
17973 started
17974 · last_updated (string) – Timestamp of last state update
17976 · info (object) – Will contain additional information about the
17978 state. For errors, this will be an object with an “error”
17979 field and string value. For success states, see below.
17980 · error_count (number) – Consecutive errors count. Indicates how
17982 many times in a row this replication has crashed. Replication
17983 will be retried with an exponential backoff based on this num‐
17984 ber. As soon as the replication succeeds this count is reset
17985 to 0. To can be used to get an idea why a particular replica‐
17986 tion is not making progress.
17987 Status Codes
17989 · 200 OK – Request completed successfully
17991 · 401 Unauthorized – CouchDB Server Administrator privileges
17993 required
17994The info field of a scheduler doc:
17996 JSON Object
17998 · revisions_checked (number) – The count of revisions
18000 which have been checked since this replication began.
18001 · missing_revisions_found (number) – The count of revi‐
18003 sions which were found on the source, but missing from
18004 the target.
18005 · docs_read (number) – The count of docs which have been
18007 read from the source.
18008 · docs_written (number) – The count of docs which have
18010 been written to the target.
18011 · changes_pending (number) – The count of changes not yet
18013 replicated.
18014 · doc_write_failures (number) – The count of docs which
18016 failed to be written to the target.
18017 · checkpointed_source_seq (object) – The source sequence
18019 id which was last successfully replicated.
18020 Request:
18022 GET /_scheduler/docs HTTP/1.1
18024 Accept: application/json
18025 Host: localhost:5984
18026 Response:
18028 HTTP/1.1 200 OK
18030 Content-Type: application/json
18031 Date: Sat, 29 Apr 2017 05:10:08 GMT
18032 Server: Server: CouchDB (Erlang/OTP)
18033 Transfer-Encoding: chunked
18034 {
18036 "docs": [
18037 {
18038 "database": "_replicator",
18039 "doc_id": "cdyno-0000001-0000002",
18040 "error_count": 0,
18041 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18042 "info": {
18043 "changes_pending": 15,
18044 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18045 "doc_write_failures": 0,
18046 "docs_read": 67,
18047 "docs_written": 67,
18048 "missing_revisions_found": 67,
18049 "revisions_checked": 67,
18050 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18051 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18052 },
18053 "last_updated": "2017-04-29T05:01:37Z",
18054 "node": "node2@127.0.0.1",
18055 "source_proxy": null,
18056 "target_proxy": null,
18057 "source": "http://myserver.com/foo",
18058 "start_time": "2017-04-29T05:01:37Z",
18059 "state": "running",
18060 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18061 },
18062 {
18063 "database": "_replicator",
18064 "doc_id": "cdyno-0000001-0000003",
18065 "error_count": 0,
18066 "id": "8f5b1bd0be6f9166ccfd36fc8be8fc22+continuous",
18067 "info": {
18068 "changes_pending": null,
18069 "checkpointed_source_seq": 0,
18070 "doc_write_failures": 0,
18071 "docs_read": 12,
18072 "docs_written": 12,
18073 "missing_revisions_found": 12,
18074 "revisions_checked": 12,
18075 "source_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg",
18076 "through_seq": "12-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE1lzgQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSexgk4yMkhITjS0wdWUBADfEJBg"
18077 },
18078 "last_updated": "2017-04-29T05:01:37Z",
18079 "node": "node1@127.0.0.1",
18080 "source_proxy": null,
18081 "target_proxy": null,
18082 "source": "http://myserver.com/foo",
18083 "start_time": "2017-04-29T05:01:37Z",
18084 "state": "running",
18085 "target": "http://adm:*****@localhost:15984/cdyno-0000003/"
18086 }
18087 ],
18088 "offset": 0,
18089 "total_rows": 2
18090 }
18091 GET /_scheduler/docs/{replicator_db}
18093 Get information about replication documents from a replicator
18094 database. The default replicator database is _replicator but
18095 other replicator databases can exist if their name ends with the
18096 suffix /_replicator.
18097 NOTE:
18099 As a convenience slashes (/) in replicator db names do not
18100 have to be escaped. So /_scheduler/docs/other/_replicator is
18101 valid and equivalent to /_scheduler/docs/other%2f_replicator
18102 Request Headers
18104 · Accept – .INDENT 2.0
18106 · application/json
18108 Response Headers
18111 · Content-Type – .INDENT 2.0
18113 · application/json
18115 Query Parameters
18118 · limit (number) – How many results to return
18120 · skip (number) – How many result to skip starting at the begin‐
18122 ning, if ordered by document ID
18123 Response JSON Object
18125 · offset (number) – How many results were skipped
18127 · total_rows (number) – Total number of replication documents.
18129 · id (string) – Replication ID, or null if state is completed or
18131 failed
18132 · state (string) – One of following states (see replica‐
18134 tor/states for descriptions): initializing, running, com‐
18135 pleted, pending, crashing, error, failed
18136 · database (string) – Database where replication document came
18138 from
18139 · doc_id (string) – Replication document ID
18141 · node (string) – Cluster node where the job is running
18143 · source (string) – Replication source
18145 · target (string) – Replication target
18147 · start_time (string) – Timestamp of when the replication was
18149 started
18150 · last_update (string) – Timestamp of last state update
18152 · info (object) – Will contain additional information about the
18154 state. For errors, this will be an object with an “error”
18155 field and string value. For success states, see below.
18156 · error_count (number) – Consecutive errors count. Indicates how
18158 many times in a row this replication has crashed. Replication
18159 will be retried with an exponential backoff based on this num‐
18160 ber. As soon as the replication succeeds this count is reset
18161 to 0. To can be used to get an idea why a particular replica‐
18162 tion is not making progress.
18163 Status Codes
18165 · 200 OK – Request completed successfully
18167 · 401 Unauthorized – CouchDB Server Administrator privileges
18169 required
18170The info field of a scheduler doc:
18172 JSON Object
18174 · revisions_checked (number) – The count of revisions
18176 which have been checked since this replication began.
18177 · missing_revisions_found (number) – The count of revi‐
18179 sions which were found on the source, but missing from
18180 the target.
18181 · docs_read (number) – The count of docs which have been
18183 read from the source.
18184 · docs_written (number) – The count of docs which have
18186 been written to the target.
18187 · changes_pending (number) – The count of changes not yet
18189 replicated.
18190 · doc_write_failures (number) – The count of docs which
18192 failed to be written to the target.
18193 · checkpointed_source_seq (object) – The source sequence
18195 id which was last successfully replicated.
18196 Request:
18198 GET /_scheduler/docs/other/_replicator HTTP/1.1
18200 Accept: application/json
18201 Host: localhost:5984
18202 Response:
18204 HTTP/1.1 200 OK
18206 Content-Type: application/json
18207 Date: Sat, 29 Apr 2017 05:10:08 GMT
18208 Server: Server: CouchDB (Erlang/OTP)
18209 Transfer-Encoding: chunked
18210 {
18212 "docs": [
18213 {
18214 "database": "other/_replicator",
18215 "doc_id": "cdyno-0000001-0000002",
18216 "error_count": 0,
18217 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18218 "info": {
18219 "changes_pending": 0,
18220 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18221 "doc_write_failures": 0,
18222 "docs_read": 67,
18223 "docs_written": 67,
18224 "missing_revisions_found": 67,
18225 "revisions_checked": 67,
18226 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18227 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18228 },
18229 "last_updated": "2017-04-29T05:01:37Z",
18230 "node": "node2@127.0.0.1",
18231 "source_proxy": null,
18232 "target_proxy": null,
18233 "source": "http://myserver.com/foo",
18234 "start_time": "2017-04-29T05:01:37Z",
18235 "state": "running",
18236 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18237 }
18238 ],
18239 "offset": 0,
18240 "total_rows": 1
18241 }
18242 GET /_scheduler/docs/{replicator_db}/{docid}
18244 NOTE:
18246 As a convenience slashes (/) in replicator db names do not
18247 have to be escaped. So /_scheduler/docs/other/_replicator is
18248 valid and equivalent to /_scheduler/docs/other%2f_replicator
18249 Request Headers
18251 · Accept – .INDENT 2.0
18253 · application/json
18255 Response Headers
18258 · Content-Type – .INDENT 2.0
18260 · application/json
18262 Response JSON Object
18265 · id (string) – Replication ID, or null if state is completed or
18267 failed
18268 · state (string) – One of following states (see replica‐
18270 tor/states for descriptions): initializing, running, com‐
18271 pleted, pending, crashing, error, failed
18272 · database (string) – Database where replication document came
18274 from
18275 · doc_id (string) – Replication document ID
18277 · node (string) – Cluster node where the job is running
18279 · source (string) – Replication source
18281 · target (string) – Replication target
18283 · start_time (string) – Timestamp of when the replication was
18285 started
18286 · last_update (string) – Timestamp of last state update
18288 · info (object) – Will contain additional information about the
18290 state. For errors, this will be an object with an “error”
18291 field and string value. For success states, see below.
18292 · error_count (number) – Consecutive errors count. Indicates how
18294 many times in a row this replication has crashed. Replication
18295 will be retried with an exponential backoff based on this num‐
18296 ber. As soon as the replication succeeds this count is reset
18297 to 0. To can be used to get an idea why a particular replica‐
18298 tion is not making progress.
18299 Status Codes
18301 · 200 OK – Request completed successfully
18303 · 401 Unauthorized – CouchDB Server Administrator privileges
18305 required
18306The info field of a scheduler doc:
18308 JSON Object
18310 · revisions_checked (number) – The count of revisions
18312 which have been checked since this replication began.
18313 · missing_revisions_found (number) – The count of revi‐
18315 sions which were found on the source, but missing from
18316 the target.
18317 · docs_read (number) – The count of docs which have been
18319 read from the source.
18320 · docs_written (number) – The count of docs which have
18322 been written to the target.
18323 · changes_pending (number) – The count of changes not yet
18325 replicated.
18326 · doc_write_failures (number) – The count of docs which
18328 failed to be written to the target.
18329 · checkpointed_source_seq (object) – .INDENT 2.0
18331 The source sequence id which was last
18333 successfully replicated.
18334 Request:
18336 GET /_scheduler/docs/other/_replicator/cdyno-0000001-0000002 HTTP/1.1
18339 Accept: application/json
18340 Host: localhost:5984
18341 Response:
18343 HTTP/1.1 200 OK
18345 Content-Type: application/json
18346 Date: Sat, 29 Apr 2017 05:10:08 GMT
18347 Server: Server: CouchDB (Erlang/OTP)
18348 Transfer-Encoding: chunked
18349 {
18351 "database": "other/_replicator",
18352 "doc_id": "cdyno-0000001-0000002",
18353 "error_count": 0,
18354 "id": "e327d79214831ca4c11550b4a453c9ba+continuous",
18355 "info": {
18356 "changes_pending": 0,
18357 "checkpointed_source_seq": "60-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYEyVygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSSpgk4yMkhITjS0wdWUBAENCJEg",
18358 "doc_write_failures": 0,
18359 "docs_read": 67,
18360 "docs_written": 67,
18361 "missing_revisions_found": 67,
18362 "revisions_checked": 67,
18363 "source_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8",
18364 "through_seq": "67-g1AAAACTeJzLYWBgYMpgTmHgz8tPSTV0MDQy1zMAQsMckEQiQ1L9____szKYE2VygQLsBsZm5pZJJpjKcRqRxwIkGRqA1H-oSepgk4yMkhITjS0wdWUBAEVKJE8"
18365 },
18366 "last_updated": "2017-04-29T05:01:37Z",
18367 "node": "node2@127.0.0.1",
18368 "source_proxy": null,
18369 "target_proxy": null,
18370 "source": "http://myserver.com/foo",
18371 "start_time": "2017-04-29T05:01:37Z",
18372 "state": "running",
18373 "target": "http://adm:*****@localhost:15984/cdyno-0000002/"
18374 }
18375 /_node/{node-name}
18377 GET /_node/{node-name}
18378 The /_node/{node-name} endpoint can be used to confirm the
18379 Erlang node name of the server that processes the request. This
18380 is most useful when accessing /_node/_local to retrieve this
18381 information. Repeatedly retrieving this information for a
18382 CouchDB endpoint can be useful to determine if a CouchDB cluster
18383 is correctly proxied through a reverse load balancer.
18384 Request Headers
18386 · Accept – .INDENT 2.0
18388 · application/json
18390 · text/plain
18392 Response Headers
18395 · Content-Type – .INDENT 2.0
18397 · application/json
18399 · text/plain; charset=utf-8
18401 Status Codes
18404 · 200 OK – Request completed successfully
18406
18408 GET /_node/_local HTTP/1.1
18410 Accept: application/json
18411 Host: localhost:5984
18412 Response:
18414 HTTP/1.1 200 OK
18416 Cache-Control: must-revalidate
18417 Content-Length: 27
18418 Content-Type: application/json
18419 Date: Tue, 28 Jan 2020 19:25:51 GMT
18420 Server: CouchDB (Erlang OTP)
18421 X-Couch-Request-ID: 5b8db6c677
18422 X-CouchDB-Body-Time: 0
18423 {"name":"node1@127.0.0.1"}
18425 /_node/{node-name}/_stats
18427 GET /_node/{node-name}/_stats
18428 The _stats resource returns a JSON object containing the statis‐
18429 tics for the running server. The object is structured with
18430 top-level sections collating the statistics for a range of
18431 entries, with each individual statistic being easily identified,
18432 and the content of each statistic is self-describing.
18433 Statistics are sampled internally on a configurable interval.
18435 When monitoring the _stats endpoint, you need to use a polling
18436 frequency of at least twice this to observe accurate results.
18437 For example, if the interval is 10 seconds, poll _stats at least
18438 every 5 seconds.
18439 The literal string _local serves as an alias for the local node
18441 name, so for all stats URLs, {node-name} may be replaced with
18442 _local, to interact with the local node’s statistics.
18443 Request Headers
18445 · Accept – .INDENT 2.0
18447 · application/json
18449 · text/plain
18451 Response Headers
18454 · Content-Type – .INDENT 2.0
18456 · application/json
18458 · text/plain; charset=utf-8
18460 Status Codes
18463 · 200 OK – Request completed successfully
18465
18467 GET /_node/_local/_stats/couchdb/request_time HTTP/1.1
18469 Accept: application/json
18470 Host: localhost:5984
18471 Response:
18473 HTTP/1.1 200 OK
18475 Cache-Control: must-revalidate
18476 Content-Length: 187
18477 Content-Type: application/json
18478 Date: Sat, 10 Aug 2013 11:41:11 GMT
18479 Server: CouchDB (Erlang/OTP)
18480 {
18482 "value": {
18483 "min": 0,
18484 "max": 0,
18485 "arithmetic_mean": 0,
18486 "geometric_mean": 0,
18487 "harmonic_mean": 0,
18488 "median": 0,
18489 "variance": 0,
18490 "standard_deviation": 0,
18491 "skewness": 0,
18492 "kurtosis": 0,
18493 "percentile": [
18494 [
18495 50,
18496 0
18497 ],
18498 [
18499 75,
18500 0
18501 ],
18502 [
18503 90,
18504 0
18505 ],
18506 [
18507 95,
18508 0
18509 ],
18510 [
18511 99,
18512 0
18513 ],
18514 [
18515 999,
18516 0
18517 ]
18518 ],
18519 "histogram": [
18520 [
18521 0,
18522 0
18523 ]
18524 ],
18525 "n": 0
18526 },
18527 "type": "histogram",
18528 "desc": "length of a request inside CouchDB without MochiWeb"
18529 }
18530The fields provide the current, minimum and maximum, and a collection of sta‐
18532tistical means and quantities. The quantity in each case is not defined, but
18533the descriptions below provide sufficient detail to determine units.
18534Statistics are reported by ‘group’. The statistics are divided into the fol‐
18536lowing top-level sections:
18537 · couch_log: Logging subsystem
18539 · couch_replicator: Replication scheduler and subsystem
18541 · couchdb: Primary CouchDB database operations
18543 · fabric: Cluster-related operations
18545 · global_changes: Global changes feed
18547 · mem3: Node membership-related statistics
18549 · pread: CouchDB file-related exceptions
18551 · rexi: Cluster internal RPC-related statistics
18553 The type of the statistic is included in the type field, and is one of
18555 the following:
18556 · counter: Monotonically increasing counter, resets on restart
18558 · histogram: Binned set of values with meaningful subdivisions. Scoped
18560 to the current collection interval.
18561 · gauge: Single numerical value that can go up and down
18563 You can also access individual statistics by quoting the statistics
18565 sections and statistic ID as part of the URL path. For example, to get
18566 the request_time statistics within the couchdb section for the target
18567 node, you can use:
18568 GET /_node/_local/_stats/couchdb/request_time HTTP/1.1
18570 This returns an entire statistics object, as with the full request, but
18572 containing only the requested individual statistic.
18573 /_node/{node-name}/_system
18575 GET /_node/{node-name}/_system
18576 The _system resource returns a JSON object containing various
18577 system-level statistics for the running server. The object is
18578 structured with top-level sections collating the statistics for
18579 a range of entries, with each individual statistic being easily
18580 identified, and the content of each statistic is self-describ‐
18581 ing.
18582 The literal string _local serves as an alias for the local node
18584 name, so for all stats URLs, {node-name} may be replaced with
18585 _local, to interact with the local node’s statistics.
18586 Request Headers
18588 · Accept – .INDENT 2.0
18590 · application/json
18592 · text/plain
18594 Response Headers
18597 · Content-Type – .INDENT 2.0
18599 · application/json
18601 · text/plain; charset=utf-8
18603 Status Codes
18606 · 200 OK – Request completed successfully
18608
18610 GET /_node/_local/_system HTTP/1.1
18612 Accept: application/json
18613 Host: localhost:5984
18614 Response:
18616 HTTP/1.1 200 OK
18618 Cache-Control: must-revalidate
18619 Content-Length: 187
18620 Content-Type: application/json
18621 Date: Sat, 10 Aug 2013 11:41:11 GMT
18622 Server: CouchDB (Erlang/OTP)
18623 {
18625 "uptime": 259,
18626 "memory": {
18627 ...
18628 }
18629 These statistics are generally intended for CouchDB developers
18631 only.
18632 /_node/{node-name}/_restart
18634 POST /_node/{node-name}/_restart
18635 This API is to facilitate integration testing only it is not
18636 meant to be used in production
18637 Status Codes
18639 · 200 OK – Request completed successfully
18641 /_search_analyze
18643 WARNING:
18644 Search endpoints require a running search plugin connected to each
18645 cluster node. See Search Plugin Installation for details.
18646 New in version 3.0.
18648 POST /_search_analyze
18651 Tests the results of Lucene analyzer tokenization on sample
18652 text.
18653 Parameters
18655 · field – Type of analyzer
18657 · text – Analyzer token you want to test
18659 Status Codes
18661 · 200 OK – Request completed successfully
18663 · 400 Bad Request – Request body is wrong (malformed or
18665 missing one of the mandatory fields)
18666 · 500 Internal Server Error – A server error (or other
18668 kind of error) occurred
18669 Request:
18671 POST /_search_analyze HTTP/1.1
18673 Host: localhost:5984
18674 Content-Type: application/json
18675 {"analyzer":"english", "text":"running"}
18677 Response:
18679 {
18681 "tokens": [
18682 "run"
18683 ]
18684 }
18685 /_utils
18687 GET /_utils
18688 Accesses the built-in Fauxton administration interface for
18689 CouchDB.
18690 Response Headers
18692 · Location – New URI location
18694 Status Codes
18696 · 301 Moved Permanently – Redirects to GET /_utils/
18698 GET /_utils/
18700 Response Headers
18702 · Content-Type – text/html
18704 · Last-Modified – Static files modification timestamp
18706 Status Codes
18708 · 200 OK – Request completed successfully
18710 /_up
18712 New in version 2.0.
18713 GET /_up
18716 Confirms that the server is up, running, and ready to respond to
18717 requests. If maintenance_mode is true or nolb, the endpoint
18718 will return a 404 response.
18719 Response Headers
18721 · Content-Type – application/json
18723 Status Codes
18725 · 200 OK – Request completed successfully
18727 · 404 Not Found – The server is unavaialble for requests
18729 at this time.
18730 Response:
18732 HTTP/1.1 200 OK
18734 Cache-Control: must-revalidate
18735 Content-Length: 16
18736 Content-Type: application/json
18737 Date: Sat, 17 Mar 2018 04:46:26 GMT
18738 Server: CouchDB/2.2.0-f999071ec (Erlang OTP/19)
18739 X-Couch-Request-ID: c57a3b2787
18740 X-CouchDB-Body-Time: 0
18741 {"status":"ok"}
18743 /_uuids
18745 Changed in version 2.0.0.
18746 GET /_uuids
18749 Requests one or more Universally Unique Identifiers (UUIDs) from
18750 the CouchDB instance. The response is a JSON object providing a
18751 list of UUIDs.
18752 Request Headers
18754 · Accept – .INDENT 2.0
18756 · application/json
18758 · text/plain
18760 Query Parameters
18763 · count (number) – Number of UUIDs to return. Default is 1.
18765 Response Headers
18767 · Content-Type – .INDENT 2.0
18769 · application/json
18771 · text/plain; charset=utf-8
18773 · ETag – Response hash
18776 Status Codes
18778 · 200 OK – Request completed successfully
18780 · 400 Bad Request – Requested more UUIDs than is allowed to
18782 retrieve
18783
18785 GET /_uuids?count=10 HTTP/1.1
18787 Accept: application/json
18788 Host: localhost:5984
18789 Response:
18791 HTTP/1.1 200 OK
18793 Content-Length: 362
18794 Content-Type: application/json
18795 Date: Sat, 10 Aug 2013 11:46:25 GMT
18796 ETag: "DGRWWQFLUDWN5MRKSLKQ425XV"
18797 Expires: Fri, 01 Jan 1990 00:00:00 GMT
18798 Pragma: no-cache
18799 Server: CouchDB (Erlang/OTP)
18800 {
18802 "uuids": [
18803 "75480ca477454894678e22eec6002413",
18804 "75480ca477454894678e22eec600250b",
18805 "75480ca477454894678e22eec6002c41",
18806 "75480ca477454894678e22eec6003b90",
18807 "75480ca477454894678e22eec6003fca",
18808 "75480ca477454894678e22eec6004bef",
18809 "75480ca477454894678e22eec600528f",
18810 "75480ca477454894678e22eec6005e0b",
18811 "75480ca477454894678e22eec6006158",
18812 "75480ca477454894678e22eec6006161"
18813 ]
18814 }
18815The UUID type is determined by the UUID algorithm setting in the CouchDB con‐
18817figuration.
18818The UUID type may be changed at any time through the Configuration API. For
18820example, the UUID type could be changed to random by sending this HTTP
18821request:
18822 PUT http://couchdb:5984/_node/nonode@nohost/_config/uuids/algorithm HTTP/1.1
18824 Content-Type: application/json
18825 Accept: */*
18826 "random"
18828 You can verify the change by obtaining a list of UUIDs:
18830 {
18832 "uuids" : [
18833 "031aad7b469956cf2826fcb2a9260492",
18834 "6ec875e15e6b385120938df18ee8e496",
18835 "cff9e881516483911aa2f0e98949092d",
18836 "b89d37509d39dd712546f9510d4a9271",
18837 "2e0dbf7f6c4ad716f21938a016e4e59f"
18838 ]
18839 }
18840 /favicon.ico
18842 GET /favicon.ico
18843 Binary content for the favicon.ico site icon.
18844 Response Headers
18846 · Content-Type – image/x-icon
18848 Status Codes
18850 · 200 OK – Request completed successfully
18852 · 404 Not Found – The requested content could not be
18854 found
18855 /_reshard
18857 New in version 2.4.
18858 GET /_reshard
18861 Returns a count of completed, failed, running, stopped, and
18862 total jobs along with the state of resharding on the cluster.
18863 Request Headers
18865 · Accept – .INDENT 2.0
18867 · application/json
18869 Response Headers
18872 · Content-Type – .INDENT 2.0
18874 · application/json
18876 Response JSON Object
18879 · state (string) – stopped or running
18881 · state_reason (string) – null or string describing additional
18883 information or reason associated with the state
18884 · completed (number) – Count of completed resharding jobs
18886 · failed (number) – Count of failed resharding jobs
18888 · running (number) – Count of running resharding jobs
18890 · stopped (number) – Count of stopped resharding jobs
18892 · total (number) – Total count of resharding jobs
18894 Status Codes
18896 · 200 OK – Request completed successfully
18898 · 401 Unauthorized – CouchDB Server Administrator privileges
18900 required
18901
18903 GET /_reshard HTTP/1.1
18905 Accept: application/json
18906 Host: localhost:5984
18907 Response:
18909 HTTP/1.1 200 OK
18911 Content-Type: application/json
18912 {
18914 "completed": 21,
18915 "failed": 0,
18916 "running": 3,
18917 "state": "running",
18918 "state_reason": null,
18919 "stopped": 0,
18920 "total": 24
18921 }
18922 GET /_reshard/state
18924 Returns the resharding state and optional information about the
18925 state.
18926 Request Headers
18928 · Accept – .INDENT 2.0
18930 · application/json
18932 Response Headers
18935 · Content-Type – .INDENT 2.0
18937 · application/json
18939 Response JSON Object
18942 · state (string) – stopped or running
18944 · state_reason (string) – Additional information or reason
18946 associated with the state
18947 Status Codes
18949 · 200 OK – Request completed successfully
18951 · 401 Unauthorized – CouchDB Server Administrator privileges
18953 required
18954
18956 GET /_reshard/state HTTP/1.1
18958 Accept: application/json
18959 Host: localhost:5984
18960 Response:
18962 HTTP/1.1 200 OK
18964 Content-Type: application/json
18965 {
18967 "reason": null,
18968 "state": "running"
18969 }
18970 PUT /_reshard/state
18972 Change the resharding state on the cluster. The states are
18973 stopped or running. This starts and stops global resharding on
18974 all the nodes of the cluster. If there are any running jobs,
18975 they will be stopped when the state changes to stopped. When the
18976 state changes back to running those job will continue running.
18977 Request Headers
18979 · Accept – .INDENT 2.0
18981 · application/json
18983 Response Headers
18986 · Content-Type – .INDENT 2.0
18988 · application/json
18990 Request JSON Object
18993 · state (string) – stopped or running
18995 · state_reason (string) – Optional string describing additional
18997 information or reason associated with the state
18998 Response JSON Object
19000 · ok (boolean) – true
19002 Status Codes
19004 · 200 OK – Request completed successfully
19006 · 400 Bad Request – Invalid request. Could be a bad or missing
19008 state name.
19009 · 401 Unauthorized – CouchDB Server Administrator privileges
19011 required
19012
19014 PUT /_reshard/state HTTP/1.1
19016 Accept: application/json
19017 Host: localhost:5984
19018 {
19020 "state": "stopped",
19021 "reason": "Rebalancing in progress"
19022 }
19023 Response:
19025 HTTP/1.1 200 OK
19027 Content-Type: application/json
19028 {
19030 "ok": true
19031 }
19032 GET /_reshard/jobs
19034 NOTE:
19036 The shape of the response and the total_rows and offset field
19037 in particular are meant to be consistent with the _sched‐
19038 uler/jobs endpoint.
19039 Request Headers
19041 · Accept – .INDENT 2.0
19043 · application/json
19045 Response Headers
19048 · Content-Type – .INDENT 2.0
19050 · application/json
19052 Response JSON Object
19055 · jobs (list) – Array of json objects, one for each resharding
19057 job. For the fields of each job see the /_reshard/job/{jobid}
19058 endpoint.
19059 · offset (number) – Offset in the list of jobs object. Currently
19061 hard-coded at 0.
19062 · total_rows (number) – Total number of resharding jobs on the
19064 cluster.
19065 Status Codes
19067 · 200 OK – Request completed successfully
19069 · 401 Unauthorized – CouchDB Server Administrator privileges
19071 required
19072
19074 GET /_reshard/jobs HTTP/1.1
19076 Accept: application/json
19077 Response:
19079 HTTP/1.1 200 OK
19081 Content-Type: application/json
19082 {
19084 "jobs": [
19085 {
19086 "history": [
19087 {
19088 "detail": null,
19089 "timestamp": "2019-03-28T15:28:02Z",
19090 "type": "new"
19091 },
19092 {
19093 "detail": "initial_copy",
19094 "timestamp": "2019-03-28T15:28:02Z",
19095 "type": "running"
19096 },
19097 ...
19098 ],
19099 "id": "001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a",
19100 "job_state": "completed",
19101 "node": "node1@127.0.0.1",
19102 "source": "shards/00000000-1fffffff/d1.1553786862",
19103 "split_state": "completed",
19104 "start_time": "2019-03-28T15:28:02Z",
19105 "state_info": {},
19106 "target": [
19107 "shards/00000000-0fffffff/d1.1553786862",
19108 "shards/10000000-1fffffff/d1.1553786862"
19109 ],
19110 "type": "split",
19111 "update_time": "2019-03-28T15:28:08Z"
19112 },
19113 ...
19114 ],
19115 "offset": 0,
19116 "total_rows": 24
19117 }
19118 GET /_reshard/jobs/{jobid}
19120 Get information about the resharding job identified by jobid.
19121 Request Headers
19123 · Accept – .INDENT 2.0
19125 · application/json
19127 Response Headers
19130 · Content-Type – .INDENT 2.0
19132 · application/json
19134 Response JSON Object
19137 · id (string) – Job ID.
19139 · type (string) – Currently only split is implemented.
19141 · job_state (string) – The running state of the job. Could be
19143 one of new, running, stopped, completed or failed.
19144 · split_state (string) – State detail specific to shard split‐
19146 ting. It indicates how far has shard splitting progressed, and
19147 can be one of new, initial_copy, topoff1, build_indices,
19148 topoff2, copy_local_docs, update_shardmap, wait_source_close,
19149 topoff3, source_delete or completed.
19150 · state_info (object) – Optional additional info associated with
19152 the current state.
19153 · source (string) – For split jobs this will be the source
19155 shard.
19156 · target (list) – For split jobs this will be a list of two or
19158 more target shards.
19159 · history (list) – List of json objects recording a job’s state
19161 transition history.
19162 Status Codes
19164 · 200 OK – Request completed successfully
19166 · 401 Unauthorized – CouchDB Server Administrator privileges
19168 required
19169
19171 GET /_reshard/jobs/001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a HTTP/1.1
19173 Accept: application/json
19174 Response:
19176 HTTP/1.1 200 OK
19178 Content-Type: application/json
19179 {
19181 "id": "001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a",
19183 "job_state": "completed",
19184 "node": "node1@127.0.0.1",
19185 "source": "shards/00000000-1fffffff/d1.1553786862",
19186 "split_state": "completed",
19187 "start_time": "2019-03-28T15:28:02Z",
19188 "state_info": {},
19189 "target": [
19190 "shards/00000000-0fffffff/d1.1553786862",
19191 "shards/10000000-1fffffff/d1.1553786862"
19192 ],
19193 "type": "split",
19194 "update_time": "2019-03-28T15:28:08Z",
19195 "history": [
19196 {
19197 "detail": null,
19198 "timestamp": "2019-03-28T15:28:02Z",
19199 "type": "new"
19200 },
19201 {
19202 "detail": "initial_copy",
19203 "timestamp": "2019-03-28T15:28:02Z",
19204 "type": "running"
19205 },
19206 ...
19207 ]
19208 }
19209 POST /_reshard/jobs/{jobid}
19211 Depending on what fields are specified in the request, one or
19212 more resharding jobs will be created. The response is a json
19213 array of results. Each result object represents a single
19214 resharding job for a particular node and range. Some of the
19215 responses could be successful and some could fail. Successful
19216 results will have the "ok": true key and and value, and failed
19217 jobs will have the "error": "{error_message}" key and value.
19218 Request Headers
19220 · Accept – .INDENT 2.0
19222 · application/json
19224 Response Headers
19227 · Content-Type – .INDENT 2.0
19229 · application/json
19231 Request JSON Object
19234 · type (string) – Type of job. Currently only "split" is
19236 accepted.
19237 · db (string) – Database to split. This is mutually exclusive
19239 with the "shard” field.
19240 · node (string) – Split shards on a particular node. This is an
19242 optional parameter. The value should be one of the nodes
19243 returned from the _membership endpoint.
19244 · range (string) – Split shards copies in the given range. The
19246 range format is hhhhhhhh-hhhhhhhh where h is a hexadecimal
19247 digit. This format is used since this is how the ranges are
19248 represented in the file system. This is parameter is optional
19249 and is mutually exclusive with the "shard" field.
19250 · shard (string) – Split a particular shard. The shard should be
19252 specified as "shards/{range}/{db}.{suffix}". Where range has
19253 the hhhhhhhh-hhhhhhhh format, db is the database name, and
19254 suffix is the shard (timestamp) creation suffix.
19255 · error (string) – Error message if a job could be not be cre‐
19257 ated.
19258 · node – Cluster node where the job was created and is running.
19260 Response JSON Object
19262 · ok (boolean) – true if job created successfully.
19264 Status Codes
19266 · 201 Created – One or more jobs were successfully created
19268 · 400 Bad Request – Invalid request. Parameter validation might
19270 have failed.
19271 · 401 Unauthorized – CouchDB Server Administrator privileges
19273 required
19274 · 404 Not Found – Db, node, range or shard was not found
19276
19278 POST /_reshard/jobs HTTP/1.1
19280 Accept: application/json
19281 Content-Type: application/json
19282 {
19284 "db": "db3",
19285 "range": "80000000-ffffffff",
19286 "type": "split"
19287 }
19288 Response:
19290 HTTP/1.1 201 Created
19292 Content-Type: application/json
19293 [
19295 {
19296 "id": "001-30d7848a6feeb826d5e3ea5bb7773d672af226fd34fd84a8fb1ca736285df557",
19297 "node": "node1@127.0.0.1",
19298 "ok": true,
19299 "shard": "shards/80000000-ffffffff/db3.1554148353"
19300 },
19301 {
19302 "id": "001-c2d734360b4cb3ff8b3feaccb2d787bf81ce2e773489eddd985ddd01d9de8e01",
19303 "node": "node2@127.0.0.1",
19304 "ok": true,
19305 "shard": "shards/80000000-ffffffff/db3.1554148353"
19306 }
19307 ]
19308 DELETE /_reshard/jobs/{jobid}
19310 If the job is running, stop the job and then remove it.
19311 Response JSON Object
19313 · ok (boolean) – true if the job was removed success‐
19315 fully.
19316 Status Codes
19318 · 200 OK – The job was removed successfully
19320 · 401 Unauthorized – CouchDB Server Administrator privi‐
19322 leges required
19323 · 404 Not Found – The job was not found
19325 Request:
19327 DELETE /_reshard/jobs/001-171d1211418996ff47bd610b1d1257fc4ca2628868def4a05e63e8f8fe50694a HTTP/1.1
19329 Response:
19331 HTTP/1.1 200 OK
19333 Content-Type: application/json
19334 {
19336 "ok": true
19337 }
19338 GET /_reshard/jobs/{jobid}/state
19340 Returns the running state of a resharding job identified by
19341 jobid.
19342 Request Headers
19344 · Accept – .INDENT 2.0
19346 · application/json
19348 Response Headers
19351 · Content-Type – .INDENT 2.0
19353 · application/json
19355 Request JSON Object
19358 · state (string) – One of new, running, stopped, completed or
19360 failed.
19361 · state_reason (string) – Additional information associated with
19363 the state
19364 Status Codes
19366 · 200 OK – Request completed successfully
19368 · 401 Unauthorized – CouchDB Server Administrator privileges
19370 required
19371 · 404 Not Found – The job was not found
19373
19375 GET /_reshard/jobs/001-b3da04f969bbd682faaab5a6c373705cbcca23f732c386bb1a608cfbcfe9faff/state HTTP/1.1
19377 Accept: application/json
19378 Host: localhost:5984
19379 Response:
19381 HTTP/1.1 200 OK
19383 Content-Type: application/json
19384 {
19386 "reason": null,
19387 "state": "running"
19388 }
19389 PUT /_reshard/jobs/{jobid}/state
19391 Change the state of a particular resharding job identified by
19392 jobid. The state can be changed from stopped to running or from
19393 running to stopped. If an individual job is stopped via this API
19394 it will stay stopped even after the global resharding state is
19395 toggled from stopped to running. If the job is already completed
19396 its state will stay completed.
19397 Request Headers
19399 · Accept – .INDENT 2.0
19401 · application/json
19403 Response Headers
19406 · Content-Type – .INDENT 2.0
19408 · application/json
19410 Request JSON Object
19413 · state (string) – stopped or running
19415 · state_reason (string) – Optional string describing additional
19417 information or reason associated with the state
19418 Response JSON Object
19420 · ok (boolean) – true
19422 Status Codes
19424 · 200 OK – Request completed successfully
19426 · 400 Bad Request – Invalid request. Could be a bad state name,
19428 for example.
19429 · 401 Unauthorized – CouchDB Server Administrator privileges
19431 required
19432 · 404 Not Found – The job was not found
19434
19436 PUT /_reshard/state/001-b3da04f969bbd682faaab5a6c373705cbcca23f732c386bb1a608cfbcfe9faff/state HTTP/1.1
19438 Accept: application/json
19439 Host: localhost:5984
19440 {
19442 "state": "stopped",
19443 "reason": "Rebalancing in progress"
19444 }
19445 Response:
19447 HTTP/1.1 200 OK
19449 Content-Type: application/json
19450 {
19452 "ok": true
19453 }
19454 Authentication
19456 Interfaces for obtaining session and authorization data.
19457 NOTE:
19459 We also strongly recommend you set up SSL to improve all authentica‐
19460 tion methods’ security.
19461 Basic Authentication
19463 Basic authentication (RFC 2617) is a quick and simple way to authenti‐
19464 cate with CouchDB. The main drawback is the need to send user creden‐
19465 tials with each request which may be insecure and could hurt operation
19466 performance (since CouchDB must compute the password hash with every
19467 request):
19468 Request:
19470 GET / HTTP/1.1
19472 Accept: application/json
19473 Authorization: Basic cm9vdDpyZWxheA==
19474 Host: localhost:5984
19475 Response:
19477 HTTP/1.1 200 OK
19479 Cache-Control: must-revalidate
19480 Content-Length: 177
19481 Content-Type: application/json
19482 Date: Mon, 03 Dec 2012 00:44:47 GMT
19483 Server: CouchDB (Erlang/OTP)
19484 {
19486 "couchdb":"Welcome",
19487 "uuid":"0a959b9b8227188afc2ac26ccdf345a6",
19488 "version":"1.3.0",
19489 "vendor": {
19490 "version":"1.3.0",
19491 "name":"The Apache Software Foundation"
19492 }
19493 }
19494 Cookie Authentication
19496 For cookie authentication (RFC 2109) CouchDB generates a token that the
19497 client can use for the next few requests to CouchDB. Tokens are valid
19498 until a timeout. When CouchDB sees a valid token in a subsequent
19499 request, it will authenticate the user by this token without requesting
19500 the password again. By default, cookies are valid for 10 minutes, but
19501 it’s adjustable. Also it’s possible to make cookies persistent.
19502 To obtain the first token and thus authenticate a user for the first
19504 time, the username and password must be sent to the _session API.
19505 /_session
19507 POST /_session
19508 Initiates new session for specified user credentials by provid‐
19509 ing Cookie value.
19510 Request Headers
19512 · Content-Type – .INDENT 2.0
19514 · application/x-www-form-urlencoded
19516 · application/json
19518 Query Parameters
19521 · next (string) – Enforces redirect after successful login to
19523 the specified location. This location is relative from server
19524 root. Optional.
19525 Form Parameters
19527 · name – User name
19529 · password – Password
19531 Response Headers
19533 · Set-Cookie – Authorization token
19535 Response JSON Object
19537 · ok (boolean) – Operation status
19539 · name (string) – Username
19541 · roles (array) – List of user roles
19543 Status Codes
19545 · 200 OK – Successfully authenticated
19547 · 302 Found – Redirect after successful authentication
19549 · 401 Unauthorized – Username or password wasn’t recognized
19551 Request:
19553 POST /_session HTTP/1.1
19555 Accept: application/json
19556 Content-Length: 24
19557 Content-Type: application/x-www-form-urlencoded
19558 Host: localhost:5984
19559 name=root&password=relax
19561 It’s also possible to send data as JSON:
19563 POST /_session HTTP/1.1
19565 Accept: application/json
19566 Content-Length: 37
19567 Content-Type: application/json
19568 Host: localhost:5984
19569 {
19571 "name": "root",
19572 "password": "relax"
19573 }
19574 Response:
19576 HTTP/1.1 200 OK
19578 Cache-Control: must-revalidate
19579 Content-Length: 43
19580 Content-Type: application/json
19581 Date: Mon, 03 Dec 2012 01:23:14 GMT
19582 Server: CouchDB (Erlang/OTP)
19583 Set-Cookie: AuthSession=cm9vdDo1MEJCRkYwMjq0LO0ylOIwShrgt8y-UkhI-c6BGw; Version=1; Path=/; HttpOnly
19584 {"ok":true,"name":"root","roles":["_admin"]}
19586 If next query parameter was provided the response will trigger
19588 redirection to the specified location in case of successful
19589 authentication:
19590 Request:
19592 POST /_session?next=/blog/_design/sofa/_rewrite/recent-posts HTTP/1.1
19594 Accept: application/json
19595 Content-Type: application/x-www-form-urlencoded
19596 Host: localhost:5984
19597 name=root&password=relax
19599 Response:
19601 HTTP/1.1 302 Moved Temporarily
19603 Cache-Control: must-revalidate
19604 Content-Length: 43
19605 Content-Type: application/json
19606 Date: Mon, 03 Dec 2012 01:32:46 GMT
19607 Location: http://localhost:5984/blog/_design/sofa/_rewrite/recent-posts
19608 Server: CouchDB (Erlang/OTP)
19609 Set-Cookie: AuthSession=cm9vdDo1MEJDMDEzRTp7Vu5GKCkTxTVxwXbpXsBARQWnhQ; Version=1; Path=/; HttpOnly
19610 {"ok":true,"name":null,"roles":["_admin"]}
19612 GET /_session
19614 Returns information about the authenticated user, including a
19615 userctx_object, the authentication method and database that were
19616 used, and a list of configured authentication handlers on the
19617 server.
19618 Query Parameters
19620 · basic (boolean) – Accept Basic Auth by requesting this
19622 resource. Optional.
19623 Response JSON Object
19625 · ok (boolean) – Operation status
19627 · userCtx (object) – User context for the current user
19629 · info (object) – Server authentication configuration
19631 Status Codes
19633 · 200 OK – Successfully authenticated.
19635 · 401 Unauthorized – Username or password wasn’t recog‐
19637 nized.
19638 Request:
19640 GET /_session HTTP/1.1
19642 Host: localhost:5984
19643 Accept: application/json
19644 Cookie: AuthSession=cm9vdDo1MEJDMDQxRDpqb-Ta9QfP9hpdPjHLxNTKg_Hf9w
19645 Response:
19647 HTTP/1.1 200 OK
19649 Cache-Control: must-revalidate
19650 Content-Length: 175
19651 Content-Type: application/json
19652 Date: Fri, 09 Aug 2013 20:27:45 GMT
19653 Server: CouchDB (Erlang/OTP)
19654 Set-Cookie: AuthSession=cm9vdDo1MjA1NTBDMTqmX2qKt1KDR--GUC80DQ6-Ew_XIw; Version=1; Path=/; HttpOnly
19655 {
19657 "info": {
19658 "authenticated": "cookie",
19659 "authentication_db": "_users",
19660 "authentication_handlers": [
19661 "cookie",
19662 "default"
19663 ]
19664 },
19665 "ok": true,
19666 "userCtx": {
19667 "name": "root",
19668 "roles": [
19669 "_admin"
19670 ]
19671 }
19672 }
19673 DELETE /_session
19675 Closes user’s session by instructing the browser to clear the
19676 cookie. This does not invalidate the session from the server’s
19677 perspective, as there is no way to do this because CouchDB cook‐
19678 ies are stateless. This means calling this endpoint is purely
19679 optional from a client perspective, and it does not protect
19680 against theft of a session cookie.
19681 Status Codes
19683 · 200 OK – Successfully close session.
19685 Request:
19687 DELETE /_session HTTP/1.1
19689 Accept: application/json
19690 Cookie: AuthSession=cm9vdDo1MjA1NEVGMDo1QXNQkqC_0Qmgrk8Fw61_AzDeXw
19691 Host: localhost:5984
19692 Response:
19694 HTTP/1.1 200 OK
19696 Cache-Control: must-revalidate
19697 Content-Length: 12
19698 Content-Type: application/json
19699 Date: Fri, 09 Aug 2013 20:30:12 GMT
19700 Server: CouchDB (Erlang/OTP)
19701 Set-Cookie: AuthSession=; Version=1; Path=/; HttpOnly
19702 {
19704 "ok": true
19705 }
19706 Proxy Authentication
19708 NOTE:
19709 To use this authentication method make sure that the {chttpd_auth,
19710 proxy_authentication_handler} value in added to the list of the
19711 active chttpd/authentication_handlers:
19712 [chttpd]
19714 authentication_handlers = {chttpd_auth, cookie_authentication_handler}, {chttpd_auth, proxy_authentication_handler}, {chttpd_auth, default_authentication_handler}
19715 Proxy authentication is very useful in case your application already
19717 uses some external authentication service and you don’t want to dupli‐
19718 cate users and their roles in CouchDB.
19719 This authentication method allows creation of a userctx_object for
19721 remotely authenticated user. By default, the client just needs to pass
19722 specific headers to CouchDB with related requests:
19723 · X-Auth-CouchDB-UserName: username;
19725 · X-Auth-CouchDB-Roles: comma-separated (,) list of user roles;
19727 · X-Auth-CouchDB-Token: authentication token. When proxy_use_secret is
19729 set (which is strongly recommended!), this header provides an HMAC of
19730 the username to authenticate and the secret token to prevent requests
19731 from untrusted sources.
19732 Request:
19734 GET /_session HTTP/1.1
19736 Host: localhost:5984
19737 Accept: application/json
19738 Content-Type: application/json; charset=utf-8
19739 X-Auth-CouchDB-Roles: users,blogger
19740 X-Auth-CouchDB-UserName: foo
19741 Response:
19743 HTTP/1.1 200 OK
19745 Cache-Control: must-revalidate
19746 Content-Length: 190
19747 Content-Type: application/json
19748 Date: Fri, 14 Jun 2013 10:16:03 GMT
19749 Server: CouchDB (Erlang/OTP)
19750 {
19752 "info": {
19753 "authenticated": "proxy",
19754 "authentication_db": "_users",
19755 "authentication_handlers": [
19756 "cookie",
19757 "proxy",
19758 "default"
19759 ]
19760 },
19761 "ok": true,
19762 "userCtx": {
19763 "name": "foo",
19764 "roles": [
19765 "users",
19766 "blogger"
19767 ]
19768 }
19769 }
19770 Note that you don’t need to request session to be authenticated by this
19772 method if all required HTTP headers are provided.
19773 Configuration
19775 The CouchDB Server Configuration API provide an interface to query and
19776 update the various configuration values within a running CouchDB
19777 instance.
19778 Accessing the local node’s configuration
19780 The literal string _local serves as an alias for the local node name,
19781 so for all configuration URLs, {node-name} may be replaced with _local,
19782 to interact with the local node’s configuration.
19783 /_node/{node-name}/_config
19785 GET /_node/{node-name}/_config
19786 Returns the entire CouchDB server configuration as a JSON struc‐
19787 ture. The structure is organized by different configuration sec‐
19788 tions, with individual values.
19789 Request Headers
19791 · Accept – .INDENT 2.0
19793 · application/json
19795 · text/plain
19797 Response Headers
19800 · Content-Type – .INDENT 2.0
19802 · application/json
19804 · text/plain; charset=utf-8
19806 Status Codes
19809 · 200 OK – Request completed successfully
19811 · 401 Unauthorized – CouchDB Server Administrator privileges
19813 required
19814
19816 GET /_node/nonode@nohost/_config HTTP/1.1
19818 Accept: application/json
19819 Host: localhost:5984
19820 Response:
19822 HTTP/1.1 200 OK
19824 Cache-Control: must-revalidate
19825 Content-Length: 4148
19826 Content-Type: application/json
19827 Date: Sat, 10 Aug 2013 12:01:42 GMT
19828 Server: CouchDB (Erlang/OTP)
19829 {
19831 "attachments": {
19832 "compressible_types": "text/*, application/javascript, application/json, application/xml",
19833 "compression_level": "8"
19834 },
19835 "couchdb": {
19836 "users_db_suffix": "_users",
19837 "database_dir": "/var/lib/couchdb",
19838 "max_attachment_chunk_size": "4294967296",
19839 "max_dbs_open": "100",
19840 "os_process_timeout": "5000",
19841 "uri_file": "/var/lib/couchdb/couch.uri",
19842 "util_driver_dir": "/usr/lib64/couchdb/erlang/lib/couch-1.5.0/priv/lib",
19843 "view_index_dir": "/var/lib/couchdb"
19844 },
19845 "chttpd": {
19846 "backlog": "512",
19847 "bind_address": "0.0.0.0",
19848 "port": "5984",
19849 "require_valid_user": "false",
19850 "socket_options": "[{sndbuf, 262144}, {nodelay, true}]",
19851 "server_options": "[{recbuf, undefined}]"
19852 },
19853 "httpd": {
19854 "allow_jsonp": "false",
19855 "authentication_handlers": "{couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}",
19856 "bind_address": "192.168.0.2",
19857 "max_connections": "2048",
19858 "port": "5984",
19859 "secure_rewrites": "true"
19860 }
19861 },
19862 "log": {
19863 "writer": "file",
19864 "file": "/var/log/couchdb/couch.log",
19865 "include_sasl": "true",
19866 "level": "info"
19867 },
19868 "query_server_config": {
19869 "reduce_limit": "true"
19870 },
19871 "replicator": {
19872 "max_http_pipeline_size": "10",
19873 "max_http_sessions": "10"
19874 },
19875 "stats": {
19876 "interval": "10"
19877 },
19878 "uuids": {
19879 "algorithm": "utc_random"
19880 }
19881 }
19882 _node/{node-name}/_config/section
19884 GET /_node/{node-name}/_config/{section}
19885 Gets the configuration structure for a single section.
19886 Parameters
19888 · section – Configuration section name
19890 Request Headers
19892 · Accept – .INDENT 2.0
19894 · application/json
19896 · text/plain
19898 Response Headers
19901 · Content-Type – .INDENT 2.0
19903 · application/json
19905 · text/plain; charset=utf-8
19907 Status Codes
19910 · 200 OK – Request completed successfully
19912 · 401 Unauthorized – CouchDB Server Administrator privileges
19914 required
19915
19917 GET /_node/nonode@nohost/_config/httpd HTTP/1.1
19919 Accept: application/json
19920 Host: localhost:5984
19921 Response:
19923 HTTP/1.1 200 OK
19925 Cache-Control: must-revalidate
19926 Content-Length: 444
19927 Content-Type: application/json
19928 Date: Sat, 10 Aug 2013 12:10:40 GMT
19929 Server: CouchDB (Erlang/OTP)
19930 {
19932 "allow_jsonp": "false",
19933 "authentication_handlers": "{couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}",
19934 "bind_address": "127.0.0.1",
19935 "default_handler": "{couch_httpd_db, handle_request}",
19936 "enable_cors": "false",
19937 "port": "5984",
19938 "secure_rewrites": "true"
19939 }
19940 /_node/node/_config/section/key
19942 GET /_node/{node-name}/_config/{section}/{key}
19943 Gets a single configuration value from within a specific config‐
19944 uration section.
19945 Parameters
19947 · section – Configuration section name
19949 · key – Configuration option name
19951 Request Headers
19953 · Accept – .INDENT 2.0
19955 · application/json
19957 · text/plain
19959 Response Headers
19962 · Content-Type – .INDENT 2.0
19964 · application/json
19966 · text/plain; charset=utf-8
19968 Status Codes
19971 · 200 OK – Request completed successfully
19973 · 401 Unauthorized – CouchDB Server Administrator privileges
19975 required
19976
19978 GET /_node/nonode@nohost/_config/log/level HTTP/1.1
19980 Accept: application/json
19981 Host: localhost:5984
19982 Response:
19984 HTTP/1.1 200 OK
19986 Cache-Control: must-revalidate
19987 Content-Length: 8
19988 Content-Type: application/json
19989 Date: Sat, 10 Aug 2013 12:12:59 GMT
19990 Server: CouchDB (Erlang/OTP)
19991 "debug"
19993 NOTE:
19995 The returned value will be the JSON of the value, which may
19996 be a string or numeric value, or an array or object. Some
19997 client environments may not parse simple strings or numeric
19998 values as valid JSON.
19999 PUT /_node/{node-name}/_config/{section}/{key}
20001 Updates a configuration value. The new value should be supplied
20002 in the request body in the corresponding JSON format. If you are
20003 setting a string value, you must supply a valid JSON string. In
20004 response CouchDB sends old value for target section key.
20005 Parameters
20007 · section – Configuration section name
20009 · key – Configuration option name
20011 Request Headers
20013 · Accept – .INDENT 2.0
20015 · application/json
20017 · text/plain
20019 · Content-Type – application/json
20022 Response Headers
20024 · Content-Type – .INDENT 2.0
20026 · application/json
20028 · text/plain; charset=utf-8
20030 Status Codes
20033 · 200 OK – Request completed successfully
20035 · 400 Bad Request – Invalid JSON request body
20037 · 401 Unauthorized – CouchDB Server Administrator privileges
20039 required
20040 · 500 Internal Server Error – Error setting configuration
20042
20044 PUT /_node/nonode@nohost/_config/log/level HTTP/1.1
20046 Accept: application/json
20047 Content-Length: 7
20048 Content-Type: application/json
20049 Host: localhost:5984
20050 "info"
20052 Response:
20054 HTTP/1.1 200 OK
20056 Cache-Control: must-revalidate
20057 Content-Length: 8
20058 Content-Type: application/json
20059 Date: Sat, 10 Aug 2013 12:12:59 GMT
20060 Server: CouchDB (Erlang/OTP)
20061 "debug"
20063 DELETE /_node/{node-name}/_config/{section}/{key}
20065 Deletes a configuration value. The returned JSON will be the
20066 value of the configuration parameter before it was deleted.
20067 Parameters
20069 · section – Configuration section name
20071 · key – Configuration option name
20073 Request Headers
20075 · Accept – .INDENT 2.0
20077 · application/json
20079 · text/plain
20081 Response Headers
20084 · Content-Type – .INDENT 2.0
20086 · application/json
20088 · text/plain; charset=utf-8
20090 Status Codes
20093 · 200 OK – Request completed successfully
20095 · 401 Unauthorized – CouchDB Server Administrator privileges
20097 required
20098 · 404 Not Found – Specified configuration option not found
20100
20102 DELETE /_node/nonode@nohost/_config/log/level HTTP/1.1
20104 Accept: application/json
20105 Host: localhost:5984
20106 Response:
20108 HTTP/1.1 200 OK
20110 Cache-Control: must-revalidate
20111 Content-Length: 7
20112 Content-Type: application/json
20113 Date: Sat, 10 Aug 2013 12:29:03 GMT
20114 Server: CouchDB (Erlang/OTP)
20115 "info"
20117 _node/{node-name}/_config/_reload
20119 New in version 3.0.
20120 POST /_node/{node-name}/_config/_reload
20123 Reloads the configuration from disk. This has a side effect of
20124 flushing any in-memory configuration changes that have not been
20125 committed to disk.
20126 Request:
20128 POST /_node/nonode@nohost/_config/_reload HTTP/1.1
20130 Host: localhost:5984
20131 Response:
20133 HTTP/1.1 200 OK
20135 Cache-Control: must-revalidate
20136 Content-Length: 12
20137 Content-Type: application/json
20138 Date: Tues, 21 Jan 2020 11:09:35
20139 Server: CouchDB/3.0.0 (Erlang OTP)
20140 {"ok":true}
20142 Databases
20144 The Database endpoint provides an interface to an entire database with
20145 in CouchDB. These are database-level, rather than document-level
20146 requests.
20147 For all these requests, the database name within the URL path should be
20149 the database name that you wish to perform the operation on. For exam‐
20150 ple, to obtain the meta information for the database recipes, you would
20151 use the HTTP request:
20152 GET /recipes
20154 For clarity, the form below is used in the URL paths:
20156 GET /db
20158 Where db is the name of any database.
20160 /db
20162 HEAD /{db}
20163 Returns the HTTP Headers containing a minimal amount of informa‐
20164 tion about the specified database. Since the response body is
20165 empty, using the HEAD method is a lightweight way to check if
20166 the database exists already or not.
20167 Parameters
20169 · db – Database name
20171 Status Codes
20173 · 200 OK – Database exists
20175 · 404 Not Found – Requested database not found
20177 Request:
20179 HEAD /test HTTP/1.1
20181 Host: localhost:5984
20182 Response:
20184 HTTP/1.1 200 OK
20186 Cache-Control: must-revalidate
20187 Content-Type: application/json
20188 Date: Mon, 12 Aug 2013 01:27:41 GMT
20189 Server: CouchDB (Erlang/OTP)
20190 GET /{db}
20192 Gets information about the specified database.
20193 Parameters
20195 · db – Database name
20197 Request Headers
20199 · Accept – .INDENT 2.0
20201 · application/json
20203 · text/plain
20205 Response Headers
20208 · Content-Type – .INDENT 2.0
20210 · application/json
20212 · text/plain; charset=utf-8
20214 Response JSON Object
20217 · cluster.n (number) – Replicas. The number of copies of every
20219 document.
20220 · cluster.q (number) – Shards. The number of range partitions.
20222 · cluster.r (number) – Read quorum. The number of consistent
20224 copies of a document that need to be read before a successful
20225 reply.
20226 · cluster.w (number) – Write quorum. The number of copies of a
20228 document that need to be written before a successful reply.
20229 · compact_running (boolean) – Set to true if the database com‐
20231 paction routine is operating on this database.
20232 · db_name (string) – The name of the database.
20234 · disk_format_version (number) – The version of the physical
20236 format used for the data when it is stored on disk.
20237 · doc_count (number) – A count of the documents in the specified
20239 database.
20240 · doc_del_count (number) – Number of deleted documents
20242 · instance_start_time (string) – Always "0". (Returned for
20244 legacy reasons.)
20245 · purge_seq (string) – An opaque string that describes the purge
20247 state of the database. Do not rely on this string for counting
20248 the number of purge operations.
20249 · sizes.active (number) – The size of live data inside the data‐
20251 base, in bytes.
20252 · sizes.external (number) – The uncompressed size of database
20254 contents in bytes.
20255 · sizes.file (number) – The size of the database file on disk in
20257 bytes. Views indexes are not included in the calculation.
20258 · update_seq (string) – An opaque string that describes the
20260 state of the database. Do not rely on this string for counting
20261 the number of updates.
20262 · props.partitioned (boolean) – (optional) If present and true,
20264 this indicates that the database is partitioned.
20265 Status Codes
20267 · 200 OK – Request completed successfully
20269 · 404 Not Found – Requested database not found
20271
20273 GET /receipts HTTP/1.1
20275 Accept: application/json
20276 Host: localhost:5984
20277 Response:
20279 HTTP/1.1 200 OK
20281 Cache-Control: must-revalidate
20282 Content-Length: 258
20283 Content-Type: application/json
20284 Date: Mon, 12 Aug 2013 01:38:57 GMT
20285 Server: CouchDB (Erlang/OTP)
20286 {
20288 "cluster": {
20289 "n": 3,
20290 "q": 8,
20291 "r": 2,
20292 "w": 2
20293 },
20294 "compact_running": false,
20295 "db_name": "receipts",
20296 "disk_format_version": 6,
20297 "doc_count": 6146,
20298 "doc_del_count": 64637,
20299 "instance_start_time": "0",
20300 "props": {},
20301 "purge_seq": 0,
20302 "sizes": {
20303 "active": 65031503,
20304 "external": 66982448,
20305 "file": 137433211
20306 },
20307 "update_seq": "292786-g1AAAAF..."
20308 }
20309 PUT /{db}
20311 Creates a new database. The database name {db} must be composed
20312 by following next rules:
20313 · Name must begin with a lowercase letter (a-z)
20315 · Lowercase characters (a-z)
20317 · Digits (0-9)
20319 · Any of the characters _, $, (, ), +, -, and /.
20321 If you’re familiar with Regular Expressions, the rules above
20323 could be written as ^[a-z][a-z0-9_$()+/-]*$.
20324 Parameters
20326 · db – Database name
20328 Query Parameters
20330 · q (integer) – Shards, aka the number of range parti‐
20332 tions. Default is 8, unless overridden in the cluster
20333 config.
20334 · n (integer) – Replicas. The number of copies of the
20336 database in the cluster. The default is 3, unless over‐
20337 ridden in the cluster config .
20338 · partitioned (boolean) – Whether to create a partitioned
20340 database. Default is false.
20341 Request Headers
20343 · Accept – .INDENT 2.0
20345 · application/json
20347 · text/plain
20349 Response Headers
20352 · Content-Type – .INDENT 2.0
20354 · application/json
20356 · text/plain; charset=utf-8
20358 · Location – Database URI location
20361 Response JSON Object
20363 · ok (boolean) – Operation status. Available in case of success
20365 · error (string) – Error type. Available if response code is 4xx
20367 · reason (string) – Error description. Available if response
20369 code is 4xx
20370 Status Codes
20372 · 201 Created – Database created successfully (quorum is met)
20374 · 202 Accepted – Accepted (at least by one node)
20376 · 400 Bad Request – Invalid database name
20378 · 401 Unauthorized – CouchDB Server Administrator privileges
20380 required
20381 · 412 Precondition Failed – Database already exists
20383
20385 PUT /db HTTP/1.1
20387 Accept: application/json
20388 Host: localhost:5984
20389 Response:
20391 HTTP/1.1 201 Created
20393 Cache-Control: must-revalidate
20394 Content-Length: 12
20395 Content-Type: application/json
20396 Date: Mon, 12 Aug 2013 08:01:45 GMT
20397 Location: http://localhost:5984/db
20398 Server: CouchDB (Erlang/OTP)
20399 {
20401 "ok": true
20402 }
20403 If we repeat the same request to CouchDB, it will response with
20405 412 since the database already exists:
20406 Request:
20408 PUT /db HTTP/1.1
20410 Accept: application/json
20411 Host: localhost:5984
20412 Response:
20414 HTTP/1.1 412 Precondition Failed
20416 Cache-Control: must-revalidate
20417 Content-Length: 95
20418 Content-Type: application/json
20419 Date: Mon, 12 Aug 2013 08:01:16 GMT
20420 Server: CouchDB (Erlang/OTP)
20421 {
20423 "error": "file_exists",
20424 "reason": "The database could not be created, the file already exists."
20425 }
20426 If an invalid database name is supplied, CouchDB returns
20428 response with 400:
20429 Request:
20431 PUT /_db HTTP/1.1
20433 Accept: application/json
20434 Host: localhost:5984
20435 Request:
20437 HTTP/1.1 400 Bad Request
20439 Cache-Control: must-revalidate
20440 Content-Length: 194
20441 Content-Type: application/json
20442 Date: Mon, 12 Aug 2013 08:02:10 GMT
20443 Server: CouchDB (Erlang/OTP)
20444 {
20446 "error": "illegal_database_name",
20447 "reason": "Name: '_db'. Only lowercase characters (a-z), digits (0-9), and any of the characters _, $, (, ), +, -, and / are allowed. Must begin with a letter."
20448 }
20449 DELETE /{db}
20451 Deletes the specified database, and all the documents and
20452 attachments contained within it.
20453 NOTE:
20455 To avoid deleting a database, CouchDB will respond with the
20456 HTTP status code 400 when the request URL includes a ?rev=
20457 parameter. This suggests that one wants to delete a document
20458 but forgot to add the document id to the URL.
20459 Parameters
20461 · db – Database name
20463 Request Headers
20465 · Accept – .INDENT 2.0
20467 · application/json
20469 · text/plain
20471 Response Headers
20474 · Content-Type – .INDENT 2.0
20476 · application/json
20478 · text/plain; charset=utf-8
20480 Response JSON Object
20483 · ok (boolean) – Operation status
20485 Status Codes
20487 · 200 OK – Database removed successfully (quorum is met and
20489 database is deleted by at least one node)
20490 · 202 Accepted – Accepted (deleted by at least one of the nodes,
20492 quorum is not met yet)
20493 · 400 Bad Request – Invalid database name or forgotten document
20495 id by accident
20496 · 401 Unauthorized – CouchDB Server Administrator privileges
20498 required
20499 · 404 Not Found – Database doesn’t exist or invalid database
20501 name
20502
20504 DELETE /db HTTP/1.1
20506 Accept: application/json
20507 Host: localhost:5984
20508 Response:
20510 HTTP/1.1 200 OK
20512 Cache-Control: must-revalidate
20513 Content-Length: 12
20514 Content-Type: application/json
20515 Date: Mon, 12 Aug 2013 08:54:00 GMT
20516 Server: CouchDB (Erlang/OTP)
20517 {
20519 "ok": true
20520 }
20521 POST /{db}
20523 Creates a new document in the specified database, using the sup‐
20524 plied JSON document structure.
20525 If the JSON structure includes the _id field, then the document
20527 will be created with the specified document ID.
20528 If the _id field is not specified, a new unique ID will be gen‐
20530 erated, following whatever UUID algorithm is configured for that
20531 server.
20532 Parameters
20534 · db – Database name
20536 Request Headers
20538 · Accept – .INDENT 2.0
20540 · application/json
20542 · text/plain
20544 · Content-Type – application/json
20547 Query Parameters
20549 · batch (string) – Stores document in batch mode Possible val‐
20551 ues: ok. Optional
20552 Response Headers
20554 · Content-Type – .INDENT 2.0
20556 · application/json
20558 · text/plain; charset=utf-8
20560 · Location – Document’s URI
20563 Response JSON Object
20565 · id (string) – Document ID
20567 · ok (boolean) – Operation status
20569 · rev (string) – Revision info
20571 Status Codes
20573 · 201 Created – Document created and stored on disk
20575 · 202 Accepted – Document data accepted, but not yet stored on
20577 disk
20578 · 400 Bad Request – Invalid database name
20580 · 401 Unauthorized – Write privileges required
20582 · 404 Not Found – Database doesn’t exist
20584 · 409 Conflict – A Conflicting Document with same ID already
20586 exists
20587
20589 POST /db HTTP/1.1
20591 Accept: application/json
20592 Content-Length: 81
20593 Content-Type: application/json
20594 {
20596 "servings": 4,
20597 "subtitle": "Delicious with fresh bread",
20598 "title": "Fish Stew"
20599 }
20600 Response:
20602 HTTP/1.1 201 Created
20604 Cache-Control: must-revalidate
20605 Content-Length: 95
20606 Content-Type: application/json
20607 Date: Tue, 13 Aug 2013 15:19:25 GMT
20608 Location: http://localhost:5984/db/ab39fe0993049b84cfa81acd6ebad09d
20609 Server: CouchDB (Erlang/OTP)
20610 {
20612 "id": "ab39fe0993049b84cfa81acd6ebad09d",
20613 "ok": true,
20614 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
20615 }
20616 Specifying the Document ID
20618 The document ID can be specified by including the _id field in the JSON
20619 of the submitted record. The following request will create the same
20620 document with the ID FishStew.
20621 Request:
20622 POST /db HTTP/1.1
20624 Accept: application/json
20625 Content-Length: 98
20626 Content-Type: application/json
20627 {
20629 "_id": "FishStew",
20630 "servings": 4,
20631 "subtitle": "Delicious with fresh bread",
20632 "title": "Fish Stew"
20633 }
20634 Response:
20636 HTTP/1.1 201 Created
20638 Cache-Control: must-revalidate
20639 Content-Length: 71
20640 Content-Type: application/json
20641 Date: Tue, 13 Aug 2013 15:19:25 GMT
20642 ETag: "1-9c65296036141e575d32ba9c034dd3ee"
20643 Location: http://localhost:5984/db/FishStew
20644 Server: CouchDB (Erlang/OTP)
20645 {
20647 "id": "FishStew",
20648 "ok": true,
20649 "rev": "1-9c65296036141e575d32ba9c034dd3ee"
20650 }
20651 Batch Mode Writes
20653 You can write documents to the database at a higher rate by using the
20654 batch option. This collects document writes together in memory (on a
20655 per-user basis) before they are committed to disk. This increases the
20656 risk of the documents not being stored in the event of a failure, since
20657 the documents are not written to disk immediately.
20658 Batch mode is not suitable for critical data, but may be ideal for
20660 applications such as log data, when the risk of some data loss due to a
20661 crash is acceptable.
20662 To use batch mode, append the batch=ok query argument to the URL of a
20664 POST /{db}, PUT /{db}/{docid}, or DELETE /{db}/{docid} request. The
20665 CouchDB server will respond with an HTTP 202 Accepted response code
20666 immediately.
20667 NOTE:
20669 Creating or updating documents with batch mode doesn’t guarantee
20670 that all documents will be successfully stored on disk. For example,
20671 individual documents may not be saved due to conflicts, rejection by
20672 validation function or by other reasons, even if overall the batch
20673 was successfully submitted.
20674 Request:
20676 POST /db?batch=ok HTTP/1.1
20678 Accept: application/json
20679 Content-Length: 98
20680 Content-Type: application/json
20681 {
20683 "_id": "FishStew",
20684 "servings": 4,
20685 "subtitle": "Delicious with fresh bread",
20686 "title": "Fish Stew"
20687 }
20688 Response:
20690 HTTP/1.1 202 Accepted
20692 Cache-Control: must-revalidate
20693 Content-Length: 28
20694 Content-Type: application/json
20695 Date: Tue, 13 Aug 2013 15:19:25 GMT
20696 Location: http://localhost:5984/db/FishStew
20697 Server: CouchDB (Erlang/OTP)
20698 {
20700 "id": "FishStew",
20701 "ok": true
20702 }
20703 /db/_all_docs
20705 GET /{db}/_all_docs
20706 Executes the built-in _all_docs view, returning all of the docu‐
20707 ments in the database. With the exception of the URL parameters
20708 (described below), this endpoint works identically to any other
20709 view. Refer to the view endpoint documentation for a complete
20710 description of the available query parameters and the format of
20711 the returned data.
20712 Parameters
20714 · db – Database name
20716 Request Headers
20718 · Content-Type – application/json
20720 Response Headers
20722 · Content-Type – .INDENT 2.0
20724 · application/json
20726 Status Codes
20729 · 200 OK – Request completed successfully
20731 · 404 Not Found – Requested database not found
20733 Request:
20735 GET /db/_all_docs HTTP/1.1
20737 Accept: application/json
20738 Host: localhost:5984
20739 Response:
20741 HTTP/1.1 200 OK
20743 Cache-Control: must-revalidate
20744 Content-Type: application/json
20745 Date: Sat, 10 Aug 2013 16:22:56 GMT
20746 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
20747 Server: CouchDB (Erlang/OTP)
20748 Transfer-Encoding: chunked
20749 {
20751 "offset": 0,
20752 "rows": [
20753 {
20754 "id": "16e458537602f5ef2a710089dffd9453",
20755 "key": "16e458537602f5ef2a710089dffd9453",
20756 "value": {
20757 "rev": "1-967a00dff5e02add41819138abb3284d"
20758 }
20759 },
20760 {
20761 "id": "a4c51cdfa2069f3e905c431114001aff",
20762 "key": "a4c51cdfa2069f3e905c431114001aff",
20763 "value": {
20764 "rev": "1-967a00dff5e02add41819138abb3284d"
20765 }
20766 },
20767 {
20768 "id": "a4c51cdfa2069f3e905c4311140034aa",
20769 "key": "a4c51cdfa2069f3e905c4311140034aa",
20770 "value": {
20771 "rev": "5-6182c9c954200ab5e3c6bd5e76a1549f"
20772 }
20773 },
20774 {
20775 "id": "a4c51cdfa2069f3e905c431114003597",
20776 "key": "a4c51cdfa2069f3e905c431114003597",
20777 "value": {
20778 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
20779 }
20780 },
20781 {
20782 "id": "f4ca7773ddea715afebc4b4b15d4f0b3",
20783 "key": "f4ca7773ddea715afebc4b4b15d4f0b3",
20784 "value": {
20785 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
20786 }
20787 }
20788 ],
20789 "total_rows": 5
20790 }
20791 POST /{db}/_all_docs
20793 POST _all_docs functionality supports identical parameters and
20794 behavior as specified in the GET /{db}/_all_docs API but allows
20795 for the query string parameters to be supplied as keys in a JSON
20796 object in the body of the POST request.
20797 Request:
20799 POST /db/_all_docs HTTP/1.1
20801 Accept: application/json
20802 Content-Length: 70
20803 Content-Type: application/json
20804 Host: localhost:5984
20805 {
20807 "keys" : [
20808 "Zingylemontart",
20809 "Yogurtraita"
20810 ]
20811 }
20812 Response:
20814 {
20816 "total_rows" : 2666,
20817 "rows" : [
20818 {
20819 "value" : {
20820 "rev" : "1-a3544d296de19e6f5b932ea77d886942"
20821 },
20822 "id" : "Zingylemontart",
20823 "key" : "Zingylemontart"
20824 },
20825 {
20826 "value" : {
20827 "rev" : "1-91635098bfe7d40197a1b98d7ee085fc"
20828 },
20829 "id" : "Yogurtraita",
20830 "key" : "Yogurtraita"
20831 }
20832 ],
20833 "offset" : 0
20834 }
20835 /db/_design_docs
20837 New in version 2.2.
20838 GET /{db}/_design_docs
20841 Returns a JSON structure of all of the design documents in a
20842 given database. The information is returned as a JSON structure
20843 containing meta information about the return structure, includ‐
20844 ing a list of all design documents and basic contents, consist‐
20845 ing the ID, revision and key. The key is the design document’s
20846 _id.
20847 Parameters
20849 · db – Database name
20851 Request Headers
20853 · Accept – .INDENT 2.0
20855 · application/json
20857 · text/plain
20859 Query Parameters
20862 · conflicts (boolean) – Includes conflicts information in
20864 response. Ignored if include_docs isn’t true. Default is
20865 false.
20866 · descending (boolean) – Return the design documents in descend‐
20868 ing by key order. Default is false.
20869 · endkey (string) – Stop returning records when the specified
20871 key is reached. Optional.
20872 · end_key (string) – Alias for endkey param.
20874 · endkey_docid (string) – Stop returning records when the speci‐
20876 fied design document ID is reached. Optional.
20877 · end_key_doc_id (string) – Alias for endkey_docid param.
20879 · include_docs (boolean) – Include the full content of the
20881 design documents in the return. Default is false.
20882 · inclusive_end (boolean) – Specifies whether the specified end
20884 key should be included in the result. Default is true.
20885 · key (string) – Return only design documents that match the
20887 specified key. Optional.
20888 · keys (string) – Return only design documents that match the
20890 specified keys. Optional.
20891 · limit (number) – Limit the number of the returned design docu‐
20893 ments to the specified number. Optional.
20894 · skip (number) – Skip this number of records before starting to
20896 return the results. Default is 0.
20897 · startkey (string) – Return records starting with the specified
20899 key. Optional.
20900 · start_key (string) – Alias for startkey param.
20902 · startkey_docid (string) – Return records starting with the
20904 specified design document ID. Optional.
20905 · start_key_doc_id (string) – Alias for startkey_docid param.
20907 · update_seq (boolean) – Response includes an update_seq value
20909 indicating which sequence id of the underlying database the
20910 view reflects. Default is false.
20911 Response Headers
20913 · Content-Type – .INDENT 2.0
20915 · application/json
20917 · text/plain; charset=utf-8
20919 · ETag – Response signature
20922 Response JSON Object
20924 · offset (number) – Offset where the design document list
20926 started
20927 · rows (array) – Array of view row objects. By default the
20929 information returned contains only the design document ID and
20930 revision.
20931 · total_rows (number) – Number of design documents in the data‐
20933 base. Note that this is not the number of rows returned in the
20934 actual query.
20935 · update_seq (number) – Current update sequence for the database
20937 Status Codes
20939 · 200 OK – Request completed successfully
20941 · 404 Not Found – Requested database not found
20943
20945 GET /db/_design_docs HTTP/1.1
20947 Accept: application/json
20948 Host: localhost:5984
20949 Response:
20951 HTTP/1.1 200 OK
20953 Cache-Control: must-revalidate
20954 Content-Type: application/json
20955 Date: Sat, 23 Dec 2017 16:22:56 GMT
20956 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
20957 Server: CouchDB (Erlang/OTP)
20958 Transfer-Encoding: chunked
20959 {
20961 "offset": 0,
20962 "rows": [
20963 {
20964 "id": "_design/ddoc01",
20965 "key": "_design/ddoc01",
20966 "value": {
20967 "rev": "1-7407569d54af5bc94c266e70cbf8a180"
20968 }
20969 },
20970 {
20971 "id": "_design/ddoc02",
20972 "key": "_design/ddoc02",
20973 "value": {
20974 "rev": "1-d942f0ce01647aa0f46518b213b5628e"
20975 }
20976 },
20977 {
20978 "id": "_design/ddoc03",
20979 "key": "_design/ddoc03",
20980 "value": {
20981 "rev": "1-721fead6e6c8d811a225d5a62d08dfd0"
20982 }
20983 },
20984 {
20985 "id": "_design/ddoc04",
20986 "key": "_design/ddoc04",
20987 "value": {
20988 "rev": "1-32c76b46ca61351c75a84fbcbceece2f"
20989 }
20990 },
20991 {
20992 "id": "_design/ddoc05",
20993 "key": "_design/ddoc05",
20994 "value": {
20995 "rev": "1-af856babf9cf746b48ae999645f9541e"
20996 }
20997 }
20998 ],
20999 "total_rows": 5
21000 }
21001 POST /{db}/_design_docs
21003 POST _design_docs functionality supports identical parameters
21004 and behavior as specified in the GET /{db}/_design_docs API but
21005 allows for the query string parameters to be supplied as keys in
21006 a JSON object in the body of the POST request.
21007 Request:
21009 POST /db/_all_docs HTTP/1.1
21011 Accept: application/json
21012 Content-Length: 70
21013 Content-Type: application/json
21014 Host: localhost:5984
21015 {
21017 "keys" : [
21018 "_design/ddoc02",
21019 "_design/ddoc05"
21020 ]
21021 }
21022 The returned JSON is the all documents structure, but with only
21024 the selected keys in the output:
21025 {
21027 "total_rows" : 5,
21028 "rows" : [
21029 {
21030 "value" : {
21031 "rev" : "1-d942f0ce01647aa0f46518b213b5628e"
21032 },
21033 "id" : "_design/ddoc02",
21034 "key" : "_design/ddoc02"
21035 },
21036 {
21037 "value" : {
21038 "rev" : "1-af856babf9cf746b48ae999645f9541e"
21039 },
21040 "id" : "_design/ddoc05",
21041 "key" : "_design/ddoc05"
21042 }
21043 ],
21044 "offset" : 0
21045 }
21046 Sending multiple queries to a database
21048 New in version 2.2.
21049 POST /{db}/_all_docs/queries
21052 Executes multiple specified built-in view queries of all docu‐
21053 ments in this database. This enables you to request multiple
21054 queries in a single request, in place of multiple POST
21055 /{db}/_all_docs requests.
21056 Parameters
21058 · db – Database name
21060 Request Headers
21062 · Content-Type – .INDENT 2.0
21064 · application/json
21066 · Accept – .INDENT 2.0
21069 · application/json
21071 Request JSON Object
21074 · queries – An array of query objects with fields for the param‐
21076 eters of each individual view query to be executed. The field
21077 names and their meaning are the same as the query parameters
21078 of a regular _all_docs request.
21079 Response Headers
21081 · Content-Type – .INDENT 2.0
21083 · application/json
21085 · text/plain; charset=utf-8
21087 · ETag – Response signature
21090 · Transfer-Encoding – chunked
21092 Response JSON Object
21094 · results (array) – An array of result objects - one for each
21096 query. Each result object contains the same fields as the
21097 response to a regular _all_docs request.
21098 Status Codes
21100 · 200 OK – Request completed successfully
21102 · 400 Bad Request – Invalid request
21104 · 401 Unauthorized – Read permission required
21106 · 404 Not Found – Specified database is missing
21108 · 500 Internal Server Error – Query execution error
21110
21112 POST /db/_all_docs/queries HTTP/1.1
21114 Content-Type: application/json
21115 Accept: application/json
21116 Host: localhost:5984
21117 {
21119 "queries": [
21120 {
21121 "keys": [
21122 "meatballs",
21123 "spaghetti"
21124 ]
21125 },
21126 {
21127 "limit": 3,
21128 "skip": 2
21129 }
21130 ]
21131 }
21132 Response:
21134 HTTP/1.1 200 OK
21136 Cache-Control: must-revalidate
21137 Content-Type: application/json
21138 Date: Wed, 20 Dec 2017 11:17:07 GMT
21139 ETag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
21140 Server: CouchDB (Erlang/OTP)
21141 Transfer-Encoding: chunked
21142 {
21144 "results" : [
21145 {
21146 "rows": [
21147 {
21148 "id": "SpaghettiWithMeatballs",
21149 "key": "meatballs",
21150 "value": 1
21151 },
21152 {
21153 "id": "SpaghettiWithMeatballs",
21154 "key": "spaghetti",
21155 "value": 1
21156 },
21157 {
21158 "id": "SpaghettiWithMeatballs",
21159 "key": "tomato sauce",
21160 "value": 1
21161 }
21162 ],
21163 "total_rows": 3
21164 },
21165 {
21166 "offset" : 2,
21167 "rows" : [
21168 {
21169 "id" : "Adukiandorangecasserole-microwave",
21170 "key" : "Aduki and orange casserole - microwave",
21171 "value" : [
21172 null,
21173 "Aduki and orange casserole - microwave"
21174 ]
21175 },
21176 {
21177 "id" : "Aioli-garlicmayonnaise",
21178 "key" : "Aioli - garlic mayonnaise",
21179 "value" : [
21180 null,
21181 "Aioli - garlic mayonnaise"
21182 ]
21183 },
21184 {
21185 "id" : "Alabamapeanutchicken",
21186 "key" : "Alabama peanut chicken",
21187 "value" : [
21188 null,
21189 "Alabama peanut chicken"
21190 ]
21191 }
21192 ],
21193 "total_rows" : 2667
21194 }
21195 ]
21196 }
21197 NOTE:
21199 The multiple queries are also supported in /db/_local_docs/queries
21200 and /db/_design_docs/queries (similar to /db/_all_docs/queries).
21201 /db/_bulk_get
21203 POST /{db}/_bulk_get
21204 This method can be called to query several documents in bulk. It
21205 is well suited for fetching a specific revision of documents, as
21206 replicators do for example, or for getting revision history.
21207 Parameters
21209 · db – Database name
21211 Request Headers
21213 · Accept – .INDENT 2.0
21215 · application/json
21217 · multipart/related
21219 · multipart/mixed
21221 · Content-Type – application/json
21224 Query Parameters
21226 · revs (boolean) – Give the revisions history
21228 Request JSON Object
21230 · docs (array) – List of document objects, with id, and option‐
21232 ally rev and atts_since
21233 Response Headers
21235 · Content-Type – .INDENT 2.0
21237 · application/json
21239 Response JSON Object
21242 · results (object) – an array of results for each requested doc‐
21244 ument/rev pair. id key lists the requested document ID, docs
21245 contains a single-item array of objects, each of which has
21246 either an error key and value describing the error, or ok key
21247 and associated value of the requested document, with the addi‐
21248 tional _revisions property that lists the parent revisions if
21249 revs=true.
21250 Status Codes
21252 · 200 OK – Request completed successfully
21254 · 400 Bad Request – The request provided invalid JSON data or
21256 invalid query parameter
21257 · 401 Unauthorized – Read permission required
21259 · 404 Not Found – Invalid database name
21261 · 415 Unsupported Media Type – Bad Content-Type value
21263
21265 POST /db/_bulk_get HTTP/1.1
21267 Accept: application/json
21268 Content-Type:application/json
21269 Host: localhost:5984
21270 {
21272 "docs": [
21273 {
21274 "id": "foo"
21275 "rev": "4-753875d51501a6b1883a9d62b4d33f91",
21276 },
21277 {
21278 "id": "foo"
21279 "rev": "1-4a7e4ae49c4366eaed8edeaea8f784ad",
21280 },
21281 {
21282 "id": "bar",
21283 }
21284 {
21285 "id": "baz",
21286 }
21287 ]
21288 }
21289 Response:
21291 HTTP/1.1 200 OK
21293 Cache-Control: must-revalidate
21294 Content-Type: application/json
21295 Date: Mon, 19 Mar 2018 15:27:34 GMT
21296 Server: CouchDB (Erlang/OTP)
21297 {
21299 "results": [
21300 {
21301 "id": "foo",
21302 "docs": [
21303 {
21304 "ok": {
21305 "_id": "foo",
21306 "_rev": "4-753875d51501a6b1883a9d62b4d33f91",
21307 "value": "this is foo",
21308 "_revisions": {
21309 "start": 4,
21310 "ids": [
21311 "753875d51501a6b1883a9d62b4d33f91",
21312 "efc54218773c6acd910e2e97fea2a608",
21313 "2ee767305024673cfb3f5af037cd2729",
21314 "4a7e4ae49c4366eaed8edeaea8f784ad"
21315 ]
21316 }
21317 }
21318 }
21319 ]
21320 },
21321 {
21322 "id": "foo",
21323 "docs": [
21324 {
21325 "ok": {
21326 "_id": "foo",
21327 "_rev": "1-4a7e4ae49c4366eaed8edeaea8f784ad",
21328 "value": "this is the first revision of foo",
21329 "_revisions": {
21330 "start": 1,
21331 "ids": [
21332 "4a7e4ae49c4366eaed8edeaea8f784ad"
21333 ]
21334 }
21335 }
21336 }
21337 ]
21338 },
21339 {
21340 "id": "bar",
21341 "docs": [
21342 {
21343 "ok": {
21344 "_id": "bar",
21345 "_rev": "2-9b71d36dfdd9b4815388eb91cc8fb61d",
21346 "baz": true,
21347 "_revisions": {
21348 "start": 2,
21349 "ids": [
21350 "9b71d36dfdd9b4815388eb91cc8fb61d",
21351 "309651b95df56d52658650fb64257b97"
21352 ]
21353 }
21354 }
21355 }
21356 ]
21357 },
21358 {
21359 "id": "baz",
21360 "docs": [
21361 {
21362 "error": {
21363 "id": "baz",
21364 "rev": "undefined",
21365 "error": "not_found",
21366 "reason": "missing"
21367 }
21368 }
21369 ]
21370 }
21371 ]
21372 }
21373 Example response with a conflicted document:
21375 Request:
21377 POST /db/_bulk_get HTTP/1.1
21379 Accept: application/json
21380 Content-Type:application/json
21381 Host: localhost:5984
21382 {
21384 "docs": [
21385 {
21386 "id": "a"
21387 }
21388 ]
21389 }
21390 Response:
21392 HTTP/1.1 200 OK
21394 Cache-Control: must-revalidate
21395 Content-Type: application/json
21396 Date: Mon, 19 Mar 2018 15:27:34 GMT
21397 Server: CouchDB (Erlang/OTP)
21398 {
21400 "results": [
21401 {
21402 "id": "a",
21403 "docs": [
21404 {
21405 "ok": {
21406 "_id": "a",
21407 "_rev": "1-23202479633c2b380f79507a776743d5",
21408 "a": 1
21409 }
21410 },
21411 {
21412 "ok": {
21413 "_id": "a",
21414 "_rev": "1-967a00dff5e02add41819138abb3284d"
21415 }
21416 }
21417 ]
21418 }
21419 ]
21420 }
21421 /db/_bulk_docs
21423 POST /{db}/_bulk_docs
21424 The bulk document API allows you to create and update multiple
21425 documents at the same time within a single request. The basic
21426 operation is similar to creating or updating a single document,
21427 except that you batch the document structure and information.
21428 When creating new documents the document ID (_id) is optional.
21430 For updating existing documents, you must provide the document
21432 ID, revision information (_rev), and new document values.
21433 In case of batch deleting documents all fields as document ID,
21435 revision information and deletion status (_deleted) are
21436 required.
21437 Parameters
21439 · db – Database name
21441 Request Headers
21443 · Accept – .INDENT 2.0
21445 · application/json
21447 · text/plain
21449 · Content-Type – application/json
21452 Request JSON Object
21454 · docs (array) – List of documents objects
21456 · new_edits (boolean) – If false, prevents the database from
21458 assigning them new revision IDs. Default is true. Optional
21459 Response Headers
21461 · Content-Type – .INDENT 2.0
21463 · application/json
21465 · text/plain; charset=utf-8
21467 Response JSON Array of Objects
21470 · id (string) – Document ID
21472 · rev (string) – New document revision token. Available if docu‐
21474 ment has saved without errors. Optional
21475 · error (string) – Error type. Optional
21477 · reason (string) – Error reason. Optional
21479 Status Codes
21481 · 201 Created – Document(s) have been created or updated
21483 · 400 Bad Request – The request provided invalid JSON data
21485 · 404 Not Found – Requested database not found
21487
21489 POST /db/_bulk_docs HTTP/1.1
21491 Accept: application/json
21492 Content-Length: 109
21493 Content-Type:application/json
21494 Host: localhost:5984
21495 {
21497 "docs": [
21498 {
21499 "_id": "FishStew"
21500 },
21501 {
21502 "_id": "LambStew",
21503 "_rev": "2-0786321986194c92dd3b57dfbfc741ce",
21504 "_deleted": true
21505 }
21506 ]
21507 }
21508 Response:
21510 HTTP/1.1 201 Created
21512 Cache-Control: must-revalidate
21513 Content-Length: 144
21514 Content-Type: application/json
21515 Date: Mon, 12 Aug 2013 00:15:05 GMT
21516 Server: CouchDB (Erlang/OTP)
21517 [
21519 {
21520 "ok": true,
21521 "id": "FishStew",
21522 "rev":" 1-967a00dff5e02add41819138abb3284d"
21523 },
21524 {
21525 "ok": true,
21526 "id": "LambStew",
21527 "rev": "3-f9c62b2169d0999103e9f41949090807"
21528 }
21529 ]
21530 Inserting Documents in Bulk
21532 Each time a document is stored or updated in CouchDB, the internal
21533 B-tree is updated. Bulk insertion provides efficiency gains in both
21534 storage space, and time, by consolidating many of the updates to inter‐
21535 mediate B-tree nodes.
21536 It is not intended as a way to perform ACID-like transactions in
21538 CouchDB, the only transaction boundary within CouchDB is a single
21539 update to a single database. The constraints are detailed in Bulk Docu‐
21540 ments Transaction Semantics.
21541 To insert documents in bulk into a database you need to supply a JSON
21543 structure with the array of documents that you want to add to the data‐
21544 base. You can either include a document ID, or allow the document ID
21545 to be automatically generated.
21546 For example, the following update inserts three new documents, two with
21548 the supplied document IDs, and one which will have a document ID gener‐
21549 ated:
21550 POST /source/_bulk_docs HTTP/1.1
21552 Accept: application/json
21553 Content-Length: 323
21554 Content-Type: application/json
21555 Host: localhost:5984
21556 {
21558 "docs": [
21559 {
21560 "_id": "FishStew",
21561 "servings": 4,
21562 "subtitle": "Delicious with freshly baked bread",
21563 "title": "FishStew"
21564 },
21565 {
21566 "_id": "LambStew",
21567 "servings": 6,
21568 "subtitle": "Serve with a whole meal scone topping",
21569 "title": "LambStew"
21570 },
21571 {
21572 "servings": 8,
21573 "subtitle": "Hand-made dumplings make a great accompaniment",
21574 "title": "BeefStew"
21575 }
21576 ]
21577 }
21578 The return type from a bulk insertion will be 201 Created, with the
21580 content of the returned structure indicating specific success or other‐
21581 wise messages on a per-document basis.
21582 The return structure from the example above contains a list of the doc‐
21584 uments created, here with the combination and their revision IDs:
21585 HTTP/1.1 201 Created
21587 Cache-Control: must-revalidate
21588 Content-Length: 215
21589 Content-Type: application/json
21590 Date: Sat, 26 Oct 2013 00:10:39 GMT
21591 Server: CouchDB (Erlang OTP)
21592 [
21594 {
21595 "id": "FishStew",
21596 "ok": true,
21597 "rev": "1-6a466d5dfda05e613ba97bd737829d67"
21598 },
21599 {
21600 "id": "LambStew",
21601 "ok": true,
21602 "rev": "1-648f1b989d52b8e43f05aa877092cc7c"
21603 },
21604 {
21605 "id": "00a271787f89c0ef2e10e88a0c0003f0",
21606 "ok": true,
21607 "rev": "1-e4602845fc4c99674f50b1d5a804fdfa"
21608 }
21609 ]
21610 For details of the semantic content and structure of the returned JSON
21612 see Bulk Documents Transaction Semantics. Conflicts and validation
21613 errors when updating documents in bulk must be handled separately; see
21614 Bulk Document Validation and Conflict Errors.
21615 Updating Documents in Bulk
21617 The bulk document update procedure is similar to the insertion proce‐
21618 dure, except that you must specify the document ID and current revision
21619 for every document in the bulk update JSON string.
21620 For example, you could send the following request:
21622 POST /recipes/_bulk_docs HTTP/1.1
21624 Accept: application/json
21625 Content-Length: 464
21626 Content-Type: application/json
21627 Host: localhost:5984
21628 {
21630 "docs": [
21631 {
21632 "_id": "FishStew",
21633 "_rev": "1-6a466d5dfda05e613ba97bd737829d67",
21634 "servings": 4,
21635 "subtitle": "Delicious with freshly baked bread",
21636 "title": "FishStew"
21637 },
21638 {
21639 "_id": "LambStew",
21640 "_rev": "1-648f1b989d52b8e43f05aa877092cc7c",
21641 "servings": 6,
21642 "subtitle": "Serve with a whole meal scone topping",
21643 "title": "LambStew"
21644 },
21645 {
21646 "_id": "BeefStew",
21647 "_rev": "1-e4602845fc4c99674f50b1d5a804fdfa",
21648 "servings": 8,
21649 "subtitle": "Hand-made dumplings make a great accompaniment",
21650 "title": "BeefStew"
21651 }
21652 ]
21653 }
21654 The return structure is the JSON of the updated documents, with the new
21656 revision and ID information:
21657 HTTP/1.1 201 Created
21659 Cache-Control: must-revalidate
21660 Content-Length: 215
21661 Content-Type: application/json
21662 Date: Sat, 26 Oct 2013 00:10:39 GMT
21663 Server: CouchDB (Erlang OTP)
21664 [
21666 {
21667 "id": "FishStew",
21668 "ok": true,
21669 "rev": "2-2bff94179917f1dec7cd7f0209066fb8"
21670 },
21671 {
21672 "id": "LambStew",
21673 "ok": true,
21674 "rev": "2-6a7aae7ac481aa98a2042718d09843c4"
21675 },
21676 {
21677 "id": "BeefStew",
21678 "ok": true,
21679 "rev": "2-9801936a42f06a16f16c30027980d96f"
21680 }
21681 ]
21682 You can optionally delete documents during a bulk update by adding the
21684 _deleted field with a value of true to each document ID/revision combi‐
21685 nation within the submitted JSON structure.
21686 The return type from a bulk insertion will be 201 Created, with the
21688 content of the returned structure indicating specific success or other‐
21689 wise messages on a per-document basis.
21690 The content and structure of the returned JSON will depend on the
21692 transaction semantics being used for the bulk update; see Bulk Docu‐
21693 ments Transaction Semantics for more information. Conflicts and valida‐
21694 tion errors when updating documents in bulk must be handled separately;
21695 see Bulk Document Validation and Conflict Errors.
21696 Bulk Documents Transaction Semantics
21698 Bulk document operations are non-atomic. This means that CouchDB does
21699 not guarantee that any individual document included in the bulk update
21700 (or insert) will be saved when you send the request. The response will
21701 contain the list of documents successfully inserted or updated during
21702 the process. In the event of a crash, some of the documents may have
21703 been successfully saved, while others lost.
21704 The response structure will indicate whether the document was updated
21706 by supplying the new _rev parameter indicating a new document revision
21707 was created. If the update failed, you will get an error of type con‐
21708 flict. For example:
21709 [
21711 {
21712 "id" : "FishStew",
21713 "error" : "conflict",
21714 "reason" : "Document update conflict."
21715 },
21716 {
21717 "id" : "LambStew",
21718 "error" : "conflict",
21719 "reason" : "Document update conflict."
21720 },
21721 {
21722 "id" : "BeefStew",
21723 "error" : "conflict",
21724 "reason" : "Document update conflict."
21725 }
21726 ]
21727 In this case no new revision has been created and you will need to sub‐
21729 mit the document update, with the correct revision tag, to update the
21730 document.
21731 Replication of documents is independent of the type of insert or
21733 update. The documents and revisions created during a bulk insert or
21734 update are replicated in the same way as any other document.
21735 Bulk Document Validation and Conflict Errors
21737 The JSON returned by the _bulk_docs operation consists of an array of
21738 JSON structures, one for each document in the original submission. The
21739 returned JSON structure should be examined to ensure that all of the
21740 documents submitted in the original request were successfully added to
21741 the database.
21742 When a document (or document revision) is not correctly committed to
21744 the database because of an error, you should check the error field to
21745 determine error type and course of action. Errors will be one of the
21746 following type:
21747 · conflict
21749 The document as submitted is in conflict. The new revision will not
21751 have been created and you will need to re-submit the document to the
21752 database.
21753 Conflict resolution of documents added using the bulk docs interface
21755 is identical to the resolution procedures used when resolving con‐
21756 flict errors during replication.
21757 · forbidden
21759 Entries with this error type indicate that the validation routine
21761 applied to the document during submission has returned an error.
21762 For example, if your validation routine includes the following:
21764 throw({forbidden: 'invalid recipe ingredient'});
21766 The error response returned will be:
21768 HTTP/1.1 201 Created
21770 Cache-Control: must-revalidate
21771 Content-Length: 80
21772 Content-Type: application/json
21773 Date: Sat, 26 Oct 2013 00:05:17 GMT
21774 Server: CouchDB (Erlang OTP)
21775 [
21777 {
21778 "id": "LambStew",
21779 "error": "forbidden",
21780 "reason": "invalid recipe ingredient"
21781 }
21782 ]
21783 /db/_find
21785 POST /{db}/_find
21786 Find documents using a declarative JSON querying syntax.
21787 Queries can use the built-in _all_docs index or custom indexes,
21788 specified using the _index endpoint.
21789 Parameters
21791 · db – Database name
21793 Request Headers
21795 · Content-Type – .INDENT 2.0
21797 · application/json
21799 Request JSON Object
21802 · selector (json) – JSON object describing criteria used to
21804 select documents. More information provided in the section on
21805 selector syntax. Required
21806 · limit (number) – Maximum number of results returned. Default
21808 is 25. Optional
21809 · skip (number) – Skip the first ‘n’ results, where ‘n’ is the
21811 value specified. Optional
21812 · sort (json) – JSON array following sort syntax. Optional
21814 · fields (array) – JSON array specifying which fields of each
21816 object should be returned. If it is omitted, the entire object
21817 is returned. More information provided in the section on
21818 filtering fields. Optional
21819 · use_index (string|array) – Instruct a query to use a specific
21821 index. Specified either as "<design_document>" or
21822 ["<design_document>", "<index_name>"]. Optional
21823 · r (number) – Read quorum needed for the result. This defaults
21825 to 1, in which case the document found in the index is
21826 returned. If set to a higher value, each document is read from
21827 at least that many replicas before it is returned in the
21828 results. This is likely to take more time than using only the
21829 document stored locally with the index. Optional, default: 1
21830 · bookmark (string) – A string that enables you to specify which
21832 page of results you require. Used for paging through result
21833 sets. Every query returns an opaque string under the bookmark
21834 key that can then be passed back in a query to get the next
21835 page of results. If any part of the selector query changes
21836 between requests, the results are undefined. Optional,
21837 default: null
21838 · update (boolean) – Whether to update the index prior to
21840 returning the result. Default is true. Optional
21841 · stable (boolean) – Whether or not the view results should be
21843 returned from a “stable” set of shards. Optional
21844 · stale (string) – Combination of update=false and stable=true
21846 options. Possible options: "ok", false (default). Optional
21847 · execution_stats (boolean) – Include execution statistics in
21849 the query response. Optional, default: ``false``
21850 Response Headers
21852 · Content-Type – application/json
21854 · Transfer-Encoding – chunked
21856 Response JSON Object
21858 · docs (object) – Array of documents matching the search. In
21860 each matching document, the fields specified in the fields
21861 part of the request body are listed, along with their values.
21862 · warning (string) – Execution warnings
21864 · execution_stats (object) – Execution statistics
21866 · bookmark (string) – An opaque string used for paging. See the
21868 bookmark field in the request (above) for usage details.
21869 Status Codes
21871 · 200 OK – Request completed successfully
21873 · 400 Bad Request – Invalid request
21875 · 401 Unauthorized – Read permission required
21877 · 404 Not Found – Requested database not found
21879 · 500 Internal Server Error – Query execution error
21881The limit and skip values are exactly as you would expect. While skip exists,
21883it is not intended to be used for paging. The reason is that the bookmark fea‐
21884ture is more efficient.
21885 Request:
21886 Example request body for finding documents using an index:
21888 POST /movies/_find HTTP/1.1
21890 Accept: application/json
21891 Content-Type: application/json
21892 Content-Length: 168
21893 Host: localhost:5984
21894 {
21896 "selector": {
21897 "year": {"$gt": 2010}
21898 },
21899 "fields": ["_id", "_rev", "year", "title"],
21900 "sort": [{"year": "asc"}],
21901 "limit": 2,
21902 "skip": 0,
21903 "execution_stats": true
21904 }
21905 Response:
21907 Example response when finding documents using an index:
21909 HTTP/1.1 200 OK
21911 Cache-Control: must-revalidate
21912 Content-Type: application/json
21913 Date: Thu, 01 Sep 2016 15:41:53 GMT
21914 Server: CouchDB (Erlang OTP)
21915 Transfer-Encoding: chunked
21916 {
21918 "docs": [
21919 {
21920 "_id": "176694",
21921 "_rev": "1-54f8e950cc338d2385d9b0cda2fd918e",
21922 "year": 2011,
21923 "title": "The Tragedy of Man"
21924 },
21925 {
21926 "_id": "780504",
21927 "_rev": "1-5f14bab1a1e9ac3ebdf85905f47fb084",
21928 "year": 2011,
21929 "title": "Drive"
21930 }
21931 ],
21932 "execution_stats": {
21933 "total_keys_examined": 0,
21934 "total_docs_examined": 200,
21935 "total_quorum_docs_examined": 0,
21936 "results_returned": 2,
21937 "execution_time_ms": 5.52
21938 }
21939 }
21940 Selector Syntax
21942 Selectors are expressed as a JSON object describing documents of inter‐
21943 est. Within this structure, you can apply conditional logic using spe‐
21944 cially named fields.
21945 Whilst selectors have some similarities with MongoDB query documents,
21947 these arise from a similarity of purpose and do not necessarily extend
21948 to commonality of function or result.
21949 Selector Basics
21951 Elementary selector syntax requires you to specify one or more fields,
21952 and the corresponding values required for those fields. This selector
21953 matches all documents whose “director” field has the value “Lars von
21954 Trier”.
21955 {
21957 "director": "Lars von Trier"
21958 }
21959 A simple selector, inspecting specific fields
21961 "selector": {
21963 "$title": "Live And Let Die"
21964 },
21965 "fields": [
21966 "title",
21967 "cast"
21968 ]
21969 You can create more complex selector expressions by combining opera‐
21971 tors. For best performance, it is best to combine ‘combination’ or
21972 ‘array logical’ operators, such as $regex, with an equality operators
21973 such as $eq, $gt, $gte, $lt, and $lte (but not $ne). For more informa‐
21974 tion about creating complex selector expressions, see creating selector
21975 expressions.
21976 Selector with 2 fields
21978 This selector matches any document with a name field containing "Paul",
21979 and that also has a location field with the value "Boston".
21980 {
21982 "name": "Paul",
21983 "location": "Boston"
21984 }
21985 Subfields
21987 A more complex selector enables you to specify the values for field of
21988 nested objects, or subfields. For example, you might use a standard
21989 JSON structure for specifying a field and subfield.
21990 Example of a field and subfield selector, using a standard JSON struc‐
21992 ture:
21993 {
21995 "imdb": {
21996 "rating": 8
21997 }
21998 }
21999 An abbreviated equivalent uses a dot notation to combine the field and
22001 subfield names into a single name.
22002 {
22004 "imdb.rating": 8
22005 }
22006 Operators
22008 Operators are identified by the use of a dollar sign ($) prefix in the
22009 name field.
22010 There are two core types of operators in the selector syntax:
22012 · Combination operators
22014 · Condition operators
22016 In general, combination operators are applied at the topmost level of
22018 selection. They are used to combine conditions, or to create combina‐
22019 tions of conditions, into one selector.
22020 Every explicit operator has the form:
22022 {"$operator": argument}
22024 A selector without an explicit operator is considered to have an
22026 implicit operator. The exact implicit operator is determined by the
22027 structure of the selector expression.
22028 Implicit Operators
22030 There are two implicit operators:
22031 · Equality
22033 · And
22035 In a selector, any field containing a JSON value, but that has no oper‐
22037 ators in it, is considered to be an equality condition. The implicit
22038 equality test applies also for fields and subfields.
22039 Any JSON object that is not the argument to a condition operator is an
22041 implicit $and operator on each field.
22042 In the below example, we use an operator to match any document, where
22044 the "year" field has a value greater than 2010:
22045 {
22047 "year": {
22048 "$gt": 2010
22049 }
22050 }
22051 In this next example, there must be a field "director" in a matching
22053 document, and the field must have a value exactly equal to "Lars von
22054 Trier".
22055 {
22057 "director": "Lars von Trier"
22058 }
22059 You can also make the equality operator explicit.
22061 {
22063 "director": {
22064 "$eq": "Lars von Trier"
22065 }
22066 }
22067 In the next example using subfields, the required field "imdb" in a
22069 matching document must also have a subfield "rating" and the subfield
22070 must have a value equal to 8.
22071 Example of implicit operator applied to a subfield test
22073 {
22075 "imdb": {
22076 "rating": 8
22077 }
22078 }
22079 Again, you can make the equality operator explicit.
22081 {
22083 "imdb": {
22084 "rating": { "$eq": 8 }
22085 }
22086 }
22087 An example of the $eq operator used with full text indexing
22089 {
22091 "selector": {
22092 "year": {
22093 "$eq": 2001
22094 }
22095 },
22096 "sort": [
22097 "title:string"
22098 ],
22099 "fields": [
22100 "title"
22101 ]
22102 }
22103 An example of the $eq operator used with database indexed on the field
22105 "year"
22106 {
22108 "selector": {
22109 "year": {
22110 "$eq": 2001
22111 }
22112 },
22113 "sort": [
22114 "year"
22115 ],
22116 "fields": [
22117 "year"
22118 ]
22119 }
22120 In this example, the field "director" must be present and contain the
22122 value "Lars von Trier" and the field "year" must exist and have the
22123 value 2003.
22124 {
22126 "director": "Lars von Trier",
22127 "year": 2003
22128 }
22129 You can make both the $and operator and the equality operator explicit.
22131 Example of using explicit $and and $eq operators
22132 {
22134 "$and": [
22135 {
22136 "director": {
22137 "$eq": "Lars von Trier"
22138 }
22139 },
22140 {
22141 "year": {
22142 "$eq": 2003
22143 }
22144 }
22145 ]
22146 }
22147 Explicit Operators
22149 All operators, apart from ‘Equality’ and ‘And’, must be stated explic‐
22150 itly.
22151 Combination Operators
22153 Combination operators are used to combine selectors. In addition to the
22154 common boolean operators found in most programming languages, there are
22155 three combination operators ($all, $elemMatch, and $allMatch) that help
22156 you work with JSON arrays.
22157 A combination operator takes a single argument. The argument is either
22159 another selector, or an array of selectors.
22160 The list of combination operators:
22162 ┌───────────┬──────────┬─────────────────────┐
22164 │Operator │ Argument │ Purpose │
22165 ├───────────┼──────────┼─────────────────────┤
22166 │$and │ Array │ Matches if all the │
22167 │ │ │ selectors in the │
22168 │ │ │ array match. │
22169 ├───────────┼──────────┼─────────────────────┤
22170 │$or │ Array │ Matches if any of │
22171 │ │ │ the selectors in │
22172 │ │ │ the array match. │
22173 │ │ │ All selectors must │
22174 │ │ │ use the same index. │
22175 ├───────────┼──────────┼─────────────────────┤
22176 │$not │ Selector │ Matches if the │
22177 │ │ │ given selector does │
22178 │ │ │ not match. │
22179 ├───────────┼──────────┼─────────────────────┤
22180 │$nor │ Array │ Matches if none of │
22181 │ │ │ the selectors in │
22182 │ │ │ the array match. │
22183 ├───────────┼──────────┼─────────────────────┤
22184 │$all │ Array │ Matches an array │
22185 │ │ │ value if it con‐ │
22186 │ │ │ tains all the ele‐ │
22187 │ │ │ ments of the argu‐ │
22188 │ │ │ ment array. │
22189 ├───────────┼──────────┼─────────────────────┤
22190 │$elemMatch │ Selector │ Matches and returns │
22191 │ │ │ all documents that │
22192 │ │ │ contain an array │
22193 │ │ │ field with at least │
22194 │ │ │ one element that │
22195 │ │ │ matches all the │
22196 │ │ │ specified query │
22197 │ │ │ criteria. │
22198 ├───────────┼──────────┼─────────────────────┤
22199 │$allMatch │ Selector │ Matches and returns │
22200 │ │ │ all documents that │
22201 │ │ │ contain an array │
22202 │ │ │ field with all its │
22203 │ │ │ elements matching │
22204 │ │ │ all the specified │
22205 │ │ │ query criteria. │
22206 └───────────┴──────────┴─────────────────────┘
22207 The $and operator
22209 $and operator used with two fields
22210 {
22212 "selector": {
22213 "$and": [
22214 {
22215 "$title": "Total Recall"
22216 },
22217 {
22218 "year": {
22219 "$in": [1984, 1991]
22220 }
22221 }
22222 ]
22223 },
22224 "fields": [
22225 "year",
22226 "title",
22227 "cast"
22228 ]
22229 }
22230 The $and operator matches if all the selectors in the array match.
22232 Below is an example using the primary index (`_all_docs`):
22233 {
22235 "$and": [
22236 {
22237 "_id": { "$gt": null }
22238 },
22239 {
22240 "year": {
22241 "$in": [2014, 2015]
22242 }
22243 }
22244 ]
22245 }
22246 The $or operator
22248 The $or operator matches if any of the selectors in the array match.
22250 Below is an example used with an index on the field "year":
22251 {
22253 "year": 1977,
22254 "$or": [
22255 { "director": "George Lucas" },
22256 { "director": "Steven Spielberg" }
22257 ]
22258 }
22259 The $not operator
22261 The $not operator matches if the given selector does not match. Below
22263 is an example used with an index on the field "year":
22264 {
22266 "year": {
22267 "$gte": 1900
22268 },
22269 "year": {
22270 "$lte": 1903
22271 },
22272 "$not": {
22273 "year": 1901
22274 }
22275 }
22276 The $nor operator
22278 The $nor operator matches if the given selector does not match. Below
22280 is an example used with an index on the field "year":
22281 {
22283 "year": {
22284 "$gte": 1900
22285 },
22286 "year": {
22287 "$lte": 1910
22288 },
22289 "$nor": [
22290 { "year": 1901 },
22291 { "year": 1905 },
22292 { "year": 1907 }
22293 ]
22294 }
22295 The $all operator
22297 The $all operator matches an array value if it contains all the ele‐
22299 ments of the argument array. Below is an example used with the primary
22300 index (_all_docs):
22301 {
22303 "_id": {
22304 "$gt": null
22305 },
22306 "genre": {
22307 "$all": ["Comedy","Short"]
22308 }
22309 }
22310 The $elemMatch operator
22312 The $elemMatch operator matches and returns all documents that contain
22314 an array field with at least one element matching the supplied query
22315 criteria. Below is an example used with the primary index (_all_docs):
22316 {
22318 "_id": { "$gt": null },
22319 "genre": {
22320 "$elemMatch": {
22321 "$eq": "Horror"
22322 }
22323 }
22324 }
22325 The $allMatch operator
22327 The $allMatch operator matches and returns all documents that contain
22329 an array field with all its elements matching the supplied query crite‐
22330 ria. Below is an example used with the primary index (_all_docs):
22331 {
22333 "_id": { "$gt": null },
22334 "genre": {
22335 "$allMatch": {
22336 "$eq": "Horror"
22337 }
22338 }
22339 }
22340 Condition Operators
22342 Condition operators are specific to a field, and are used to evaluate
22343 the value stored in that field. For instance, the basic $eq operator
22344 matches when the specified field contains a value that is equal to the
22345 supplied argument.
22346 The basic equality and inequality operators common to most programming
22348 languages are supported. In addition, some ‘meta’ condition operators
22349 are available. Some condition operators accept any valid JSON content
22350 as the argument. Other condition operators require the argument to be
22351 in a specific JSON format.
22352 ┌──────────────┬──────────┬──────────────────┬──────────────────┐
22354 │Operator type │ Operator │ Argument │ Purpose │
22355 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22356 │(In)equality │ $lt │ Any JSON │ The field is │
22357 │ │ │ │ less than the │
22358 │ │ │ │ argument │
22359 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22360 │ │ $lte │ Any JSON │ The field is │
22361 │ │ │ │ less than or │
22362 │ │ │ │ equal to the │
22363 │ │ │ │ argument. │
22364 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22365 │ │ $eq │ Any JSON │ The field is │
22366 │ │ │ │ equal to the │
22367 │ │ │ │ argument │
22368 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22369 │ │ $ne │ Any JSON │ The field is not │
22370 │ │ │ │ equal to the │
22371 │ │ │ │ argument. │
22372 └──────────────┴──────────┴──────────────────┴──────────────────┘
22373 │ │ $gte │ Any JSON │ The field is │
22375 │ │ │ │ greater than or │
22376 │ │ │ │ equal to the │
22377 │ │ │ │ argument. │
22378 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22379 │ │ $gt │ Any JSON │ The field is │
22380 │ │ │ │ greater than the │
22381 │ │ │ │ to the argument. │
22382 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22383 │Object │ $exists │ Boolean │ Check whether │
22384 │ │ │ │ the field exists │
22385 │ │ │ │ or not, regard‐ │
22386 │ │ │ │ less of its │
22387 │ │ │ │ value. │
22388 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22389 │ │ $type │ String │ Check the docu‐ │
22390 │ │ │ │ ment field’s │
22391 │ │ │ │ type. Valid │
22392 │ │ │ │ values are │
22393 │ │ │ │ "null", "bool‐ │
22394 │ │ │ │ ean", "number", │
22395 │ │ │ │ "string", │
22396 │ │ │ │ "array", and │
22397 │ │ │ │ "object". │
22398 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22399 │Array │ $in │ Array of JSON │ The document │
22400 │ │ │ values │ field must exist │
22401 │ │ │ │ in the list pro‐ │
22402 │ │ │ │ vided. │
22403 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22404 │ │ $nin │ Array of JSON │ The document │
22405 │ │ │ values │ field not must │
22406 │ │ │ │ exist in the │
22407 │ │ │ │ list provided. │
22408 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22409 │ │ $size │ Integer │ Special condi‐ │
22410 │ │ │ │ tion to match │
22411 │ │ │ │ the length of an │
22412 │ │ │ │ array field in a │
22413 │ │ │ │ document. │
22414 │ │ │ │ Non-array fields │
22415 │ │ │ │ cannot match │
22416 │ │ │ │ this condition. │
22417 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22418 │Miscellaneous │ $mod │ [Divisor, │ Divisor and │
22419 │ │ │ Remainder] │ Remainder are │
22420 │ │ │ │ both positive or │
22421 │ │ │ │ negative inte‐ │
22422 │ │ │ │ gers. Non-inte‐ │
22423 │ │ │ │ ger values │
22424 │ │ │ │ result in a 404. │
22425 │ │ │ │ Matches docu‐ │
22426 │ │ │ │ ments where │
22427 │ │ │ │ field % Divisor │
22428 │ │ │ │ == Remainder is │
22429 │ │ │ │ true, and only │
22430 │ │ │ │ when the docu‐ │
22431 │ │ │ │ ment field is an │
22432 │ │ │ │ integer. │
22433 ├──────────────┼──────────┼──────────────────┼──────────────────┤
22434 │ │ $regex │ String │ A regular │
22435 │ │ │ │ expression pat‐ │
22436 │ │ │ │ tern to match │
22437 │ │ │ │ against the doc‐ │
22438 │ │ │ │ ument field. │
22439 │ │ │ │ Only matches │
22440 │ │ │ │ when the field │
22441 │ │ │ │ is a string │
22442 │ │ │ │ value and │
22443 │ │ │ │ matches the sup‐ │
22444 │ │ │ │ plied regular │
22445 │ │ │ │ expression. The │
22446 │ │ │ │ matching algo‐ │
22447 │ │ │ │ rithms are based │
22448 │ │ │ │ on the Perl Com‐ │
22449 │ │ │ │ patible Regular │
22450 │ │ │ │ Expression │
22451 │ │ │ │ (PCRE) library. │
22452 │ │ │ │ For more infor‐ │
22453 │ │ │ │ mation about │
22454 │ │ │ │ what is imple‐ │
22455 │ │ │ │ mented, see the │
22456 │ │ │ │ see the Erlang │
22457 │ │ │ │ Regular Expres‐ │
22458 │ │ │ │ sion │
22459 └──────────────┴──────────┴──────────────────┴──────────────────┘
22460 WARNING:
22462 Regular expressions do not work with indexes, so they should not be
22463 used to filter large data sets. They can, however, be used to
22464 restrict a partial index.
22465 Creating Selector Expressions
22467 We have seen examples of combining selector expressions, such as using
22468 explicit $and and $eq operators.
22469 In general, whenever you have an operator that takes an argument, that
22471 argument can itself be another operator with arguments of its own. This
22472 enables us to build up more complex selector expressions.
22473 However, only equality operators such as $eq, $gt, $gte, $lt, and $lte
22475 (but not $ne) can be used as the basis of a query. You should include
22476 at least one of these in a selector.
22477 For example, if you try to perform a query that attempts to match all
22479 documents that have a field called afieldname containing a value that
22480 begins with the letter A, this will trigger a warning because no index
22481 could be used and the database performs a full scan of the primary
22482 index:
22483 Request
22484 POST /movies/_find HTTP/1.1
22486 Accept: application/json
22487 Content-Type: application/json
22488 Content-Length: 112
22489 Host: localhost:5984
22490 {
22492 "selector": {
22493 "afieldname": {"$regex": "^A"}
22494 }
22495 }
22496 Response:
22498 HTTP/1.1 200 OK
22500 Cache-Control: must-revalidate
22501 Content-Type: application/json
22502 Date: Thu, 01 Sep 2016 17:25:51 GMT
22503 Server: CouchDB (Erlang OTP)
22504 Transfer-Encoding: chunked
22505 {
22507 "warning":"no matching index found, create an index to optimize
22508 query time",
22509 "docs":[
22510 ]
22511 }
22512 WARNING:
22514 It’s always recommended that you create an appropriate index when
22515 deploying in production.
22516 Most selector expressions work exactly as you would expect for the
22518 given operator. But it is not always the case: for example, comparison
22519 of strings is done with ICU and can can give surprising results if you
22520 were expecting ASCII ordering. See views/collation for more details.
22521 Sort Syntax
22523 The sort field contains a list of field name and direction pairs,
22524 expressed as a basic array. The first field name and direction pair is
22525 the topmost level of sort. The second pair, if provided, is the next
22526 level of sort.
22527 The field can be any field, using dotted notation if desired for
22529 sub-document fields.
22530 The direction value is "asc" for ascending, and "desc" for descending.
22532 If you omit the direction value, the default "asc" is used.
22533 Example, sorting by 2 fields:
22535 [{"fieldName1": "desc"}, {"fieldName2": "desc" }]
22537 Example, sorting by 2 fields, assuming default direction for both :
22539 ["fieldNameA", "fieldNameB"]
22541 A typical requirement is to search for some content using a selector,
22543 then to sort the results according to the specified field, in the
22544 required direction.
22545 To use sorting, ensure that:
22547 · At least one of the sort fields is included in the selector.
22549 ·
22551 There is an index already defined, with all the sort fields in the
22553 same
22554 order.
22555 · Each object in the sort array has a single key.
22557 If an object in the sort array does not have a single key, the result‐
22559 ing sort order is implementation specific and might change.
22560 Find does not support multiple fields with different sort orders, so
22562 the directions must be either all ascending or all descending.
22563 For field names in text search sorts, it is sometimes necessary for a
22565 field type to be specified, for example:
22566 { "<fieldname>:string": "asc"}
22568 If possible, an attempt is made to discover the field type based on the
22570 selector. In ambiguous cases the field type must be provided explic‐
22571 itly.
22572 The sorting order is undefined when fields contain different data
22574 types. This is an important difference between text and view indexes.
22575 Sorting behavior for fields with different data types might change in
22576 future versions.
22577 A simple query, using sorting:
22578 {
22580 "selector": {"Actor_name": "Robert De Niro"},
22581 "sort": [{"Actor_name": "asc"}, {"Movie_runtime": "asc"}]
22582 }
22583 Filtering Fields
22585 It is possible to specify exactly which fields are returned for a docu‐
22586 ment when selecting from a database. The two advantages are:
22587 ·
22589 Your results are limited to only those parts of the document that are
22591 required for your application.
22592 · A reduction in the size of the response.
22594 The fields returned are specified as an array.
22596 Only the specified filter fields are included, in the response. There
22598 is no automatic inclusion of the _id or other metadata fields when a
22599 field list is included.
22600 Example of selective retrieval of fields from matching documents:
22602 {
22604 "selector": { "Actor_name": "Robert De Niro" },
22605 "fields": ["Actor_name", "Movie_year", "_id", "_rev"]
22606 }
22607 Pagination
22609 Mango queries support pagination via the bookmark field. Every _find
22610 response contains a bookmark - a token that CouchDB uses to determine
22611 where to resume from when subsequent queries are made. To get the next
22612 set of query results, add the bookmark that was received in the previ‐
22613 ous response to your next request. Remember to keep the selector the
22614 same, otherwise you will receive unexpected results. To paginate back‐
22615 wards, you can use a previous bookmark to return the previous set of
22616 results.
22617 Note that the presence of a bookmark doesn’t guarantee that there are
22619 more results. You can to test whether you have reached the end of the
22620 result set by comparing the number of results returned with the page
22621 size requested - if results returned < limit, there are no more.
22622 Execution Statistics
22624 Find can return basic execution statistics for a specific request. Com‐
22625 bined with the _explain endpoint, this should provide some insight as
22626 to whether indexes are being used effectively.
22627 The execution statistics currently include:
22629 ┌───────────────────────────┬────────────────────────────┐
22631 │Field │ Description │
22632 ├───────────────────────────┼────────────────────────────┤
22633 │total_keys_examined │ Number of index keys exam‐ │
22634 │ │ ined. Currently always 0. │
22635 ├───────────────────────────┼────────────────────────────┤
22636 │total_docs_examined │ Number of documents │
22637 │ │ fetched from the database │
22638 │ │ / index, equivalent to │
22639 │ │ using include_docs=true in │
22640 │ │ a view. These may then be │
22641 │ │ filtered in-memory to fur‐ │
22642 │ │ ther narrow down the │
22643 │ │ result set based on the │
22644 │ │ selector. │
22645 ├───────────────────────────┼────────────────────────────┤
22646 │total_quorum_docs_examined │ Number of documents │
22647 │ │ fetched from the database │
22648 │ │ using an out-of-band docu‐ │
22649 │ │ ment fetch. This is only │
22650 │ │ non-zero when read quorum │
22651 │ │ > 1 is specified in the │
22652 │ │ query parameters. │
22653 ├───────────────────────────┼────────────────────────────┤
22654 │results_returned │ Number of results returned │
22655 │ │ from the query. Ideally │
22656 │ │ this should not be signif‐ │
22657 │ │ icantly lower than the │
22658 │ │ total documents / keys │
22659 │ │ examined. │
22660 ├───────────────────────────┼────────────────────────────┤
22661 │execution_time_ms │ Total execution time in │
22662 │ │ milliseconds as measured │
22663 │ │ by the database. │
22664 └───────────────────────────┴────────────────────────────┘
22665 /db/_index
22667 Mango is a declarative JSON querying language for CouchDB databases.
22668 Mango wraps several index types, starting with the Primary Index
22669 out-of-the-box. Mango indexes, with index type json, are built using
22670 MapReduce Views.
22671 POST /{db}/_index
22673 Create a new index on a database
22674 Parameters
22676 · db – Database name
22678 Request Headers
22680 · Content-Type – .INDENT 2.0
22682 · application/json
22684 Query Parameters
22687 · index (json) – JSON object describing the index to create.
22689 · ddoc (string) – Name of the design document in which the index
22691 will be created. By default, each index will be created in its
22692 own design document. Indexes can be grouped into design docu‐
22693 ments for efficiency. However, a change to one index in a
22694 design document will invalidate all other indexes in the same
22695 document (similar to views). Optional
22696 · name (string) – Name of the index. If no name is provided, a
22698 name will be generated automatically. Optional
22699 · type (string) – Can be "json" or "text". Defaults to json.
22701 Geospatial indexes will be supported in the future. Optional
22702 Text indexes are supported via a third party library Optional
22703 · partial_filter_selector (json) – A selector to apply to docu‐
22705 ments at indexing time, creating a partial index. Optional
22706 · partitioned (boolean) – Determines whether a JSON index is
22708 partitioned or global. The default value of partitioned is the
22709 partitioned property of the database. To create a global index
22710 on a partitioned database, specify false for the "partitioned"
22711 field. If you specify true for the "partitioned" field on an
22712 unpartitioned database, an error occurs.
22713 Response Headers
22715 · Content-Type – application/json
22717 · Transfer-Encoding – chunked
22719 Response JSON Object
22721 · result (string) – Flag to show whether the index was created
22723 or one already exists. Can be “created” or “exists”.
22724 · id (string) – Id of the design document the index was created
22726 in.
22727 · name (string) – Name of the index created.
22729 Status Codes
22731 · 200 OK – Index created successfully or already exists
22733 · 400 Bad Request – Invalid request
22735 · 401 Unauthorized – Admin permission required
22737 · 500 Internal Server Error – Execution error
22739
22741The index object is a JSON array of field names following the sort syntax.
22743Nested fields are also allowed, e.g. “person.name”.
22744Example of creating a new index for the field called foo:
22746 Request:
22747 POST /db/_index HTTP/1.1
22749 Content-Type: application/json
22750 Content-Length: 116
22751 Host: localhost:5984
22752 {
22754 "index": {
22755 "fields": ["foo"]
22756 },
22757 "name" : "foo-index",
22758 "type" : "json"
22759 }
22760 The returned JSON confirms the index has been created:
22762 Response:
22763 HTTP/1.1 200 OK
22765 Cache-Control: must-revalidate
22766 Content-Length: 96
22767 Content-Type: application/json
22768 Date: Thu, 01 Sep 2016 18:17:48 GMT
22769 Server: CouchDB (Erlang OTP/18)
22770 {
22772 "result":"created",
22773 "id":"_design/a5f4711fc9448864a13c81dc71e660b524d7410c",
22774 "name":"foo-index"
22775 }
22776 Example index creation using all available query parameters
22778 {
22780 "selector": {
22781 "year": {
22782 "$gt": 2010
22783 }
22784 },
22785 "fields": ["_id", "_rev", "year", "title"],
22786 "sort": [{"year": "asc"}],
22787 "limit": 10,
22788 "skip": 0
22789 }
22790 By default, a JSON index will include all documents that have the
22792 indexed fields present, including those which have null values.
22793 Partial Indexes
22795 Partial indexes allow documents to be filtered at indexing time, poten‐
22796 tially offering significant performance improvements for query selec‐
22797 tors that don’t map cleanly to a range query on an index.
22798 Let’s look at an example query:
22800 {
22802 "selector": {
22803 "status": {
22804 "$ne": "archived"
22805 },
22806 "type": "user"
22807 }
22808 }
22809 Without a partial index, this requires a full index scan to find all
22811 the documents of "type":"user" that do not have a status of "archived".
22812 This is because a normal index can only be used to match contiguous
22813 rows, and the "$ne" operator cannot guarantee that.
22814 To improve response times, we can create an index which excludes docu‐
22816 ments where "status": { "$ne": "archived" } at index time using the
22817 "partial_filter_selector" field:
22818 POST /db/_index HTTP/1.1
22820 Content-Type: application/json
22821 Content-Length: 144
22822 Host: localhost:5984
22823 {
22825 "index": {
22826 "partial_filter_selector": {
22827 "status": {
22828 "$ne": "archived"
22829 }
22830 },
22831 "fields": ["type"]
22832 },
22833 "ddoc" : "type-not-archived",
22834 "type" : "json"
22835 }
22836 Partial indexes are not currently used by the query planner unless
22838 specified by a "use_index" field, so we need to modify the original
22839 query:
22840 {
22842 "selector": {
22843 "status": {
22844 "$ne": "archived"
22845 },
22846 "type": "user"
22847 },
22848 "use_index": "type-not-archived"
22849 }
22850 Technically, we don’t need to include the filter on the "status" field
22852 in the query selector - the partial index ensures this is always true -
22853 but including it makes the intent of the selector clearer and will make
22854 it easier to take advantage of future improvements to query planning
22855 (e.g. automatic selection of partial indexes).
22856 GET /{db}/_index
22858 When you make a GET request to /db/_index, you get a list of all
22859 indexes in the database. In addition to the information avail‐
22860 able through this API, indexes are also stored in design docu‐
22861 ments <index-functions>. Design documents are regular documents
22862 that have an ID starting with _design/. Design documents can be
22863 retrieved and modified in the same way as any other document,
22864 although this is not necessary when using Mango.
22865 Parameters
22867 · db – Database name.
22869 Response Headers
22871 · Content-Type – application/json
22873 · Transfer-Encoding – chunked
22875 Response JSON Object
22877 · total_rows (number) – Number of indexes
22879 · indexes (object) – Array of index definitions
22881 Status Codes
22883 · 200 OK – Success
22885 · 400 Bad Request – Invalid request
22887 · 401 Unauthorized – Read permission required
22889 · 500 Internal Server Error – Execution error
22891 Format of index objects:
22893 ·
22895 ddoc: ID of the design document the index belongs to.
22897 This ID
22898 can be used to retrieve the design document con‐
22899 taining the index, by making a GET request to
22900 /db/ddoc, where ddoc is the value of this field.
22901 · name: Name of the index.
22903 ·
22905 type: Type of the index. Currently “json” is the only
22907 supported type.
22908 ·
22910 def: Definition of the index, containing the indexed
22912 fields
22913 and the sort order: ascending or descending.
22914 Request:
22916 GET /db/_index HTTP/1.1
22918 Accept: application/json
22919 Host: localhost:5984
22920 Response:
22922 HTTP/1.1 200 OK
22924 Cache-Control: must-revalidate
22925 Content-Length: 238
22926 Content-Type: application/json
22927 Date: Thu, 01 Sep 2016 18:17:48 GMT
22928 Server: CouchDB (Erlang OTP/18)
22929 {
22931 "total_rows": 2,
22932 "indexes": [
22933 {
22934 "ddoc": null,
22935 "name": "_all_docs",
22936 "type": "special",
22937 "def": {
22938 "fields": [
22939 {
22940 "_id": "asc"
22941 }
22942 ]
22943 }
22944 },
22945 {
22946 "ddoc": "_design/a5f4711fc9448864a13c81dc71e660b524d7410c",
22947 "name": "foo-index",
22948 "type": "json",
22949 "def": {
22950 "fields": [
22951 {
22952 "foo": "asc"
22953 }
22954 ]
22955 }
22956 }
22957 ]
22958 }
22959 DELETE /{db}/_index/{designdoc}/json/{name}
22961 Parameters
22963 · db – Database name.
22965 · designdoc – Design document name.
22967 · name – Index name.
22969 Response Headers
22971 · Content-Type – application/json
22973 Response JSON Object
22975 · ok (string) – “true” if successful.
22977 Status Codes
22979 · 200 OK – Success
22981 · 400 Bad Request – Invalid request
22983 · 401 Unauthorized – Writer permission required
22985 · 404 Not Found – Index not found
22987 · 500 Internal Server Error – Execution error
22989 Request:
22991 DELETE /db/_index/_design/a5f4711fc9448864a13c81dc71e660b524d7410c/json/foo-index HTTP/1.1
22993 Accept: */*
22994 Host: localhost:5984
22995 Response:
22997 HTTP/1.1 200 OK
22999 Cache-Control: must-revalidate
23000 Content-Length: 12
23001 Content-Type: application/json
23002 Date: Thu, 01 Sep 2016 19:21:40 GMT
23003 Server: CouchDB (Erlang OTP/18)
23004 {
23006 "ok": true
23007 }
23008 /db/_explain
23010 POST /{db}/_explain
23011 Shows which index is being used by the query. Parameters are
23012 the same as _find
23013 Parameters
23015 · db – Database name
23017 Request Headers
23019 · Content-Type – application/json
23021 Response Headers
23023 · Content-Type – application/json
23025 · Transfer-Encoding – chunked
23027 Response JSON Object
23029 · dbname (string) – Name of database
23031 · index (object) – Index used to fulfill the query
23033 · selector (object) – Query selector used
23035 · opts (object) – Query options used
23037 · limit (number) – Limit parameter used
23039 · skip (number) – Skip parameter used
23041 · fields (array) – Fields to be returned by the query
23043 · range (object) – Range parameters passed to the under‐
23045 lying view
23046 Status Codes
23048 · 200 OK – Request completed successfully
23050 · 400 Bad Request – Invalid request
23052 · 401 Unauthorized – Read permission required
23054 · 500 Internal Server Error – Execution error
23056 Request:
23058 POST /movies/_explain HTTP/1.1
23060 Accept: application/json
23061 Content-Type: application/json
23062 Content-Length: 168
23063 Host: localhost:5984
23064 {
23066 "selector": {
23067 "year": {"$gt": 2010}
23068 },
23069 "fields": ["_id", "_rev", "year", "title"],
23070 "sort": [{"year": "asc"}],
23071 "limit": 2,
23072 "skip": 0
23073 }
23074 Response:
23076 HTTP/1.1 200 OK
23078 Cache-Control: must-revalidate
23079 Content-Type: application/json
23080 Date: Thu, 01 Sep 2016 15:41:53 GMT
23081 Server: CouchDB (Erlang OTP)
23082 Transfer-Encoding: chunked
23083 {
23085 "dbname": "movies",
23086 "index": {
23087 "ddoc": "_design/0d61d9177426b1e2aa8d0fe732ec6e506f5d443c",
23088 "name": "0d61d9177426b1e2aa8d0fe732ec6e506f5d443c",
23089 "type": "json",
23090 "def": {
23091 "fields": [
23092 {
23093 "year": "asc"
23094 }
23095 ]
23096 }
23097 },
23098 "selector": {
23099 "year": {
23100 "$gt": 2010
23101 }
23102 },
23103 "opts": {
23104 "use_index": [],
23105 "bookmark": "nil",
23106 "limit": 2,
23107 "skip": 0,
23108 "sort": {},
23109 "fields": [
23110 "_id",
23111 "_rev",
23112 "year",
23113 "title"
23114 ],
23115 "r": [
23116 49
23117 ],
23118 "conflicts": false
23119 },
23120 "limit": 2,
23121 "skip": 0,
23122 "fields": [
23123 "_id",
23124 "_rev",
23125 "year",
23126 "title"
23127 ],
23128 "range": {
23129 "start_key": [
23130 2010
23131 ],
23132 "end_key": [
23133 {}
23134 ]
23135 }
23136 }
23137 Index selection
23139 _find chooses which index to use for responding to a query, unless you
23140 specify an index at query time.
23141 The query planner looks at the selector section and finds the index
23143 with the closest match to operators and fields used in the query. If
23144 there are two or more json type indexes that match, the index with the
23145 smallest number of fields in the index is preferred. If there are
23146 still two or more candidate indexes, the index with the first alphabet‐
23147 ical name is chosen.
23148 NOTE:
23150 It’s good practice to specify indexes explicitly in your queries.
23151 This prevents existing queries being affected by new indexes that
23152 might get added in a production environment.
23153 /db/_shards
23155 New in version 2.0.
23156 GET /{db}/_shards
23159 The response will contain a list of database shards. Each shard
23160 will have its internal database range, and the nodes on which
23161 replicas of those shards are stored.
23162 Parameters
23164 · db – Database name
23166 Request Headers
23168 · Accept – .INDENT 2.0
23170 · application/json
23172 · text/plain
23174 Response Headers
23177 · Content-Type – .INDENT 2.0
23179 · application/json
23181 · text/plain; charset=utf-8
23183 Response JSON Object
23186 · shards (object) – Mapping of shard ranges to individual shard
23188 replicas on each node in the cluster
23189 Status Codes
23191 · 200 OK – Request completed successfully
23193 · 400 Bad Request – Invalid database name
23195 · 401 Unauthorized – Read privilege required
23197 · 415 Unsupported Media Type – Bad Content-Type value
23199 · 500 Internal Server Error – Internal server error or timeout
23201
23203 GET /db/_shards HTTP/1.1
23205 Accept: */*
23206 Host: localhost:5984
23207 Response:
23209 HTTP/1.1 200 OK
23211 Cache-Control: must-revalidate
23212 Content-Length: 621
23213 Content-Type: application/json
23214 Date: Fri, 18 Jan 2019 19:55:14 GMT
23215 Server: CouchDB/2.4.0 (Erlang OTP/19)
23216 {
23218 "shards": {
23219 "00000000-1fffffff": [
23220 "couchdb@node1.example.com",
23221 "couchdb@node2.example.com",
23222 "couchdb@node3.example.com"
23223 ],
23224 "20000000-3fffffff": [
23225 "couchdb@node1.example.com",
23226 "couchdb@node2.example.com",
23227 "couchdb@node3.example.com"
23228 ],
23229 "40000000-5fffffff": [
23230 "couchdb@node1.example.com",
23231 "couchdb@node2.example.com",
23232 "couchdb@node3.example.com"
23233 ],
23234 "60000000-7fffffff": [
23235 "couchdb@node1.example.com",
23236 "couchdb@node2.example.com",
23237 "couchdb@node3.example.com"
23238 ],
23239 "80000000-9fffffff": [
23240 "couchdb@node1.example.com",
23241 "couchdb@node2.example.com",
23242 "couchdb@node3.example.com"
23243 ],
23244 "a0000000-bfffffff": [
23245 "couchdb@node1.example.com",
23246 "couchdb@node2.example.com",
23247 "couchdb@node3.example.com"
23248 ],
23249 "c0000000-dfffffff": [
23250 "couchdb@node1.example.com",
23251 "couchdb@node2.example.com",
23252 "couchdb@node3.example.com"
23253 ],
23254 "e0000000-ffffffff": [
23255 "couchdb@node1.example.com",
23256 "couchdb@node2.example.com",
23257 "couchdb@node3.example.com"
23258 ]
23259 }
23260 }
23261 /db/_shards/doc
23263 GET /{db}/_shards/{docid}
23264 Returns information about the specific shard into which a given
23265 document has been stored, along with information about the nodes
23266 on which that shard has a replica.
23267 Parameters
23269 · db – Database name
23271 · docid – Document ID
23273 Request Headers
23275 · Accept – .INDENT 2.0
23277 · application/json
23279 · text/plain
23281 Response Headers
23284 · Content-Type – .INDENT 2.0
23286 · application/json
23288 · text/plain; charset=utf-8
23290 Response JSON Object
23293 · range (string) – The shard range in which the document is
23295 stored
23296 · nodes (array) – List of nodes serving a replica of the shard
23298 Status Codes
23300 · 200 OK – Request completed successfully
23302 · 401 Unauthorized – Read privilege required
23304 · 404 Not Found – Database or document not found
23306 · 500 Internal Server Error – Internal server error or timeout
23308
23310 HTTP/1.1 200 OK
23312 Cache-Control: must-revalidate
23313 Content-Length: 94
23314 Content-Type: application/json
23315 Date: Fri, 18 Jan 2019 20:08:07 GMT
23316 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23317 Response:
23319 HTTP/1.1 200 OK
23321 Cache-Control: must-revalidate
23322 Content-Length: 94
23323 Content-Type: application/json
23324 Date: Fri, 18 Jan 2019 20:26:33 GMT
23325 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23326 {
23328 "range": "e0000000-ffffffff",
23329 "nodes": [
23330 "node1@127.0.0.1",
23331 "node2@127.0.0.1",
23332 "node3@127.0.0.1"
23333 ]
23334 }
23335 /db/_sync_shards
23337 New in version 2.3.1.
23338 POST /{db}/_sync_shards
23341 For the given database, force-starts internal shard synchroniza‐
23342 tion for all replicas of all database shards.
23343 This is typically only used when performing cluster maintenance,
23345 such as moving a shard.
23346 Parameters
23348 · db – Database name
23350 Request Headers
23352 · Accept – .INDENT 2.0
23354 · application/json
23356 · text/plain
23358 Response Headers
23361 · Content-Type – .INDENT 2.0
23363 · application/json
23365 · text/plain; charset=utf-8
23367 Response JSON Object
23370 · ok (boolean) – Operation status. Available in case of success
23372 · error (string) – Error type. Available if response code is 4xx
23374 · reason (string) – Error description. Available if response
23376 code is 4xx
23377 Status Codes
23379 · 202 Accepted – Request accepted
23381 · 400 Bad Request – Invalid database name
23383 · 401 Unauthorized – CouchDB Server Administrator privileges
23385 required
23386 · 404 Not Found – Database not found
23388 · 500 Internal Server Error – Internal server error or timeout
23390
23392 POST /db/_sync_shards HTTP/1.1
23394 Host: localhost:5984
23395 Accept: */*
23396 Response:
23398 HTTP/1.1 202 Accepted
23400 Cache-Control: must-revalidate
23401 Content-Length: 12
23402 Content-Type: application/json
23403 Date: Fri, 18 Jan 2019 20:19:23 GMT
23404 Server: CouchDB/2.3.0-9d4cb03c2 (Erlang OTP/19)
23405 X-Couch-Request-ID: 14f0b8d252
23406 X-CouchDB-Body-Time: 0
23407 {
23409 "ok": true
23410 }
23411
23413 Admins may want to bump their [mem3] sync_concurrency value to a
23414 larger figure for the duration of the shards sync.
23415 /db/_changes
23417 GET /{db}/_changes
23418 Returns a sorted list of changes made to documents in the data‐
23419 base, in time order of application, can be obtained from the
23420 database’s _changes resource. Only the most recent change for a
23421 given document is guaranteed to be provided, for example if a
23422 document has had fields added, and then deleted, an API client
23423 checking for changes will not necessarily receive the intermedi‐
23424 ate state of added documents.
23425 This can be used to listen for update and modifications to the
23427 database for post processing or synchronization, and for practi‐
23428 cal purposes, a continuously connected _changes feed is a rea‐
23429 sonable approach for generating a real-time log for most appli‐
23430 cations.
23431 Parameters
23433 · db – Database name
23435 Request Headers
23437 · Accept – .INDENT 2.0
23439 · application/json
23441 · text/event-stream
23443 · text/plain
23445 · Last-Event-ID – ID of the last events received by the server
23448 on a previous connection. Overrides since query parameter.
23449 Query Parameters
23451 · doc_ids (array) – List of document IDs to filter the changes
23453 feed as valid JSON array. Used with _doc_ids filter. Since
23454 length of URL is limited, it is better to use POST
23455 /{db}/_changes instead.
23456 · conflicts (boolean) – Includes conflicts information in
23458 response. Ignored if include_docs isn’t true. Default is
23459 false.
23460 · descending (boolean) – Return the change results in descending
23462 sequence order (most recent change first). Default is false.
23463 · feed (string) – .INDENT 2.0
23465 · normal Specifies Normal Polling Mode. All past changes are
23467 returned immediately. Default.
23468 · longpoll Specifies Long Polling Mode. Waits until at least one
23470 change has occurred, sends the change, then closes the connec‐
23471 tion. Most commonly used in conjunction with since=now, to
23472 wait for the next change.
23473 · continuous Sets Continuous Mode. Sends a line of JSON per
23475 event. Keeps the socket open until timeout.
23476 · eventsource Sets Event Source Mode. Works the same as Continu‐
23478 ous Mode, but sends the events in EventSource format.
23479 · filter (string) – Reference to a filter function from a design docu‐
23482 ment that will filter whole stream emitting only filtered events. See
23483 the section Change Notifications in the book CouchDB The Definitive
23484 Guide for more information.
23485 · heartbeat (number) – Period in milliseconds after which an empty line
23487 is sent in the results. Only applicable for longpoll, continuous, and
23488 eventsource feeds. Overrides any timeout to keep the feed alive
23489 indefinitely. Default is 60000. May be true to use default value.
23490 · include_docs (boolean) – Include the associated document with each
23492 result. If there are conflicts, only the winning revision is
23493 returned. Default is false.
23494 · attachments (boolean) – Include the Base64-encoded content of attach‐
23496 ments in the documents that are included if include_docs is true.
23497 Ignored if include_docs isn’t true. Default is false.
23498 · att_encoding_info (boolean) – Include encoding information in attach‐
23500 ment stubs if include_docs is true and the particular attachment is
23501 compressed. Ignored if include_docs isn’t true. Default is false.
23502 · last-event-id (number) – Alias of Last-Event-ID header.
23504 · limit (number) – Limit number of result rows to the specified value
23506 (note that using 0 here has the same effect as 1).
23507 · since – Start the results from the change immediately after the given
23509 update sequence. Can be valid update sequence or now value. Default
23510 is 0.
23511 · style (string) – Specifies how many revisions are returned in the
23513 changes array. The default, main_only, will only return the current
23514 “winning” revision; all_docs will return all leaf revisions (includ‐
23515 ing conflicts and deleted former conflicts).
23516 · timeout (number) – Maximum period in milliseconds to wait for a
23518 change before the response is sent, even if there are no results.
23519 Only applicable for longpoll or continuous feeds. Default value is
23520 specified by httpd/changes_timeout configuration option. Note that
23521 60000 value is also the default maximum timeout to prevent undetected
23522 dead connections.
23523 · view (string) – Allows to use view functions as filters. Documents
23525 counted as “passed” for view filter in case if map function emits at
23526 least one record for them. See _view for more info.
23527 · seq_interval (number) – When fetching changes in a batch, setting the
23529 seq_interval parameter tells CouchDB to only calculate the update seq
23530 with every Nth result returned. By setting seq_interval=<batch size>
23531 , where <batch size> is the number of results requested per batch,
23532 load can be reduced on the source CouchDB database; computing the seq
23533 value across many shards (esp. in highly-sharded databases) is expen‐
23534 sive in a heavily loaded CouchDB cluster.
23535 Response Headers
23537 · Cache-Control – no-cache if changes feed is eventsource
23539 · Content-Type – .INDENT 2.0
23541 · application/json
23543 · text/event-stream
23545 · text/plain; charset=utf-8
23547 · ETag – Response hash if changes feed is normal
23550 · Transfer-Encoding – chunked
23552 Response JSON Object
23554 · last_seq (json) – Last change update sequence
23556 · pending (number) – Count of remaining items in the feed
23558 · results (array) – Changes made to a database
23560 Status Codes
23562 · 200 OK – Request completed successfully
23564 · 400 Bad Request – Bad request
23566The results field of database changes:
23568 JSON Object
23570 · changes (array) – List of document’s leaves with single
23572 field rev.
23573 · id (string) – Document ID.
23575 · seq (json) – Update sequence.
23577 · deleted (bool) – true if the document is deleted.
23579 Request:
23581 GET /db/_changes?style=all_docs HTTP/1.1
23583 Accept: application/json
23584 Host: localhost:5984
23585 Response:
23587 HTTP/1.1 200 OK
23589 Cache-Control: must-revalidate
23590 Content-Type: application/json
23591 Date: Mon, 12 Aug 2013 00:54:58 GMT
23592 ETag: "6ASLEKEMSRABT0O5XY9UPO9Z"
23593 Server: CouchDB (Erlang/OTP)
23594 Transfer-Encoding: chunked
23595 {
23597 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23598 "pending": 0,
23599 "results": [
23600 {
23601 "changes": [
23602 {
23603 "rev": "2-7051cbe5c8faecd085a3fa619e6e6337"
23604 }
23605 ],
23606 "id": "6478c2ae800dfc387396d14e1fc39626",
23607 "seq": "3-g1AAAAG3eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcjfGaQZmaUmmZClM8gZhyAmHGfsG0PICrBPmQC22ZqbGRqamyIqSsLAAArcXo"
23608 },
23609 {
23610 "changes": [
23611 {
23612 "rev": "3-7379b9e515b161226c6559d90c4dc49f"
23613 }
23614 ],
23615 "deleted": true,
23616 "id": "5bbc9ca465f1b0fcd62362168a7c8831",
23617 "seq": "4-g1AAAAHXeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoUtxoYGZkZG5uS4NY8FiDJ0ACkgAbNx2cfROUCiMr9CJ8ZpJkZpaaZEOUziBkHIGbcJ2zbA4hKsA-ZwLaZGhuZmhobYurKAgCz33kh"
23618 },
23619 {
23620 "changes": [
23621 {
23622 "rev": "6-460637e73a6288cb24d532bf91f32969"
23623 },
23624 {
23625 "rev": "5-eeaa298781f60b7bcae0c91bdedd1b87"
23626 }
23627 ],
23628 "id": "729eb57437745e506b333068fff665ae",
23629 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA"
23630 }
23631 ]
23632 }
23633Changed in version 0.11.0: added include_docs parameter
23635Changed in version 1.2.0: added view parameter and special value _view for
23639Changed in version 1.3.0: since parameter could take now value to start listen
23642changes since current seq number.
23643Changed in version 1.3.0: eventsource feed type added.
23646Changed in version 1.4.0: Support Last-Event-ID header.
23649Changed in version 1.6.0: added attachments and att_encoding_info parameters
23652Changed in version 2.0.0: update sequences can be any valid json object, added
23656
23659 If the specified replicas of the shards in any given since value are
23660 unavailable, alternative replicas are selected, and the last known
23661 checkpoint between them is used. If this happens, you might see
23662 changes again that you have previously seen. Therefore, an applica‐
23663 tion making use of the _changes feed should be ‘idempotent’, that
23664 is, able to receive the same data multiple times, safely.
23665 NOTE:
23667 Cloudant Sync and PouchDB already optimize the replication process
23668 by setting seq_interval parameter to the number of results expected
23669 per batch. This parameter increases throughput by reducing latency
23670 between sequential requests in bulk document transfers. This has
23671 resulted in up to a 20% replication performance improvement in
23672 highly-sharded databases.
23673 WARNING:
23675 Using the attachments parameter to include attachments in the
23676 changes feed is not recommended for large attachment sizes. Also
23677 note that the Base64-encoding that is used leads to a 33% overhead
23678 (i.e. one third) in transfer size for attachments.
23679 WARNING:
23681 The results returned by _changes are partially ordered. In other
23682 words, the order is not guaranteed to be preserved for multiple
23683 calls.
23684 POST /{db}/_changes
23686 Requests the database changes feed in the same way as GET
23687 /{db}/_changes does, but is widely used with ?filter=_doc_ids
23688 query parameter and allows one to pass a larger list of document
23689 IDs to filter.
23690 Request:
23692 POST /recipes/_changes?filter=_doc_ids HTTP/1.1
23694 Accept: application/json
23695 Content-Length: 40
23696 Content-Type: application/json
23697 Host: localhost:5984
23698 {
23700 "doc_ids": [
23701 "SpaghettiWithMeatballs"
23702 ]
23703 }
23704 Response:
23706 HTTP/1.1 200 OK
23708 Cache-Control: must-revalidate
23709 Content-Type: application/json
23710 Date: Sat, 28 Sep 2013 07:23:09 GMT
23711 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
23712 Server: CouchDB (Erlang OTP)
23713 Transfer-Encoding: chunked
23714 {
23716 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV8_o5i",
23717 "pending": 0,
23718 "results": [
23719 {
23720 "changes": [
23721 {
23722 "rev": "13-bcb9d6388b60fd1e960d9ec4e8e3f29e"
23723 }
23724 ],
23725 "id": "SpaghettiWithMeatballs",
23726 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA"
23727 }
23728 ]
23729 }
23730 Changes Feeds
23732 Polling
23733 By default all changes are immediately returned within the JSON body:
23734 GET /somedatabase/_changes HTTP/1.1
23736 {"results":[
23738 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P__7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcTVnkAovI-YZUPICpBvs0CAN1eY_c","id":"fresh","changes":[{"rev":"1-967a00dff5e02add41819138abb3284d"}]},
23739 {"seq":"3-g1AAAAG3eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcjfGaQZmaUmmZClM8gZhyAmHGfsG0PICrBPmQC22ZqbGRqamyIqSsLAAArcXo","id":"updated","changes":[{"rev":"2-7051cbe5c8faecd085a3fa619e6e6337CFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEB-yD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA","id":"deleted","changes":[{"rev":"2-eec205a9d413992850a6e32678485900"}],"deleted":true}
23740 ],
23741 "last_seq":"5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23742 "pending": 0}
23743 results is the list of changes in sequential order. New and changed
23745 documents only differ in the value of the rev; deleted documents
23746 include the "deleted": true attribute. (In the style=all_docs mode,
23747 deleted applies only to the current/winning revision. The other revi‐
23748 sions listed might be deleted even if there is no deleted property; you
23749 have to GET them individually to make sure.)
23750 last_seq is the update sequence of the last update returned (Equivalent
23752 to the last item in the results).
23753 Sending a since param in the query string skips all changes up to and
23755 including the given update sequence:
23756 GET /somedatabase/_changes?since=4-g1AAAAHXeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBMZc4EC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HqQ_kQG3qgSQqnoUtxoYGZkZG5uS4NY8FiDJ0ACkgAbNx2cfROUCiMr9CJ8ZpJkZpaaZEOUziBkHIGbcJ2zbA4hKsA-ZwLaZGhuZmhobYurKAgCz33kh HTTP/1.1
23758 The return structure for normal and longpoll modes is a JSON array of
23760 changes objects, and the last update sequence.
23761 In the return format for continuous mode, the server sends a CRLF (car‐
23763 riage-return, linefeed) delimited line for each change. Each line con‐
23764 tains the JSON object described above.
23765 You can also request the full contents of each document change (instead
23767 of just the change notification) by using the include_docs parameter.
23768 {
23770 "last_seq": "5-g1AAAAIreJyVkEsKwjAURZ-toI5cgq5A0sQ0OrI70XyppcaRY92J7kR3ojupaSPUUgotgRd4yTlwbw4A0zRUMLdnpaMkwmyF3Ily9xBwEIuiKLI05KOTW0wkV4rruP29UyGWbordzwKVxWBNOGMKZhertDlarbr5pOT3DV4gudUC9-MPJX9tpEAYx4TQASns2E24ucuJ7rXJSL1BbEgf3vTwpmedCZkYa7Pulck7Xt7x_usFU2aIHOD4eEfVTVA5KMGUkqhNZV-8_o5i",
23771 "pending": 0,
23772 "results": [
23773 {
23774 "changes": [
23775 {
23776 "rev": "2-eec205a9d413992850a6e32678485900"
23777 }
23778 ],
23779 "deleted": true,
23780 "id": "deleted",
23781 "seq": "5-g1AAAAIReJyVkE0OgjAQRkcwUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloRid3MMkEUoJHbXbOxVy6arc_SxQWQzRVHCuYHaxSpuj1aqbj0t-3-AlSrZakn78oeSvjRSIkIhSNiCFHbsKN3c50b02mURvEByD296eNOzzoRMRLRZ98rkHS_veGcC_nR-fGe1gaCaxihhjOI2lX0BhniHaA",
23782 }
23783 ]
23784 }
23785 Long Polling
23787 The longpoll feed, probably most applicable for a browser, is a more
23788 efficient form of polling that waits for a change to occur before the
23789 response is sent. longpoll avoids the need to frequently poll CouchDB
23790 to discover nothing has changed!
23791 The request to the server will remain open until a change is made on
23793 the database and is subsequently transferred, and then the connection
23794 will close. This is low load for both server and client.
23795 The response is basically the same JSON as is sent for the normal feed.
23797 Because the wait for a change can be significant you can set a timeout
23799 before the connection is automatically closed (the timeout argument).
23800 You can also set a heartbeat interval (using the heartbeat query argu‐
23801 ment), which sends a newline to keep the connection active.
23802 Continuous
23804 Continually polling the CouchDB server is not ideal - setting up new
23805 HTTP connections just to tell the client that nothing happened puts
23806 unnecessary strain on CouchDB.
23807 A continuous feed stays open and connected to the database until
23809 explicitly closed and changes are sent to the client as they happen,
23810 i.e. in near real-time.
23811 As with the longpoll feed type you can set both the timeout and heart‐
23813 beat intervals to ensure that the connection is kept open for new
23814 changes and updates.
23815 The continuous feed’s response is a little different than the other
23817 feed types to simplify the job of the client - each line of the
23818 response is either empty or a JSON object representing a single change,
23819 as found in the normal feed’s results.
23820 GET /somedatabase/_changes?feed=continuous HTTP/1.1
23822 {"seq":"1-g1AAAAF9eJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MSGXAqSVIAkkn2IFUZzIkMuUAee5pRqnGiuXkKA2dpXkpqWmZeagpu_Q4g_fGEbEkAqaqH2sIItsXAyMjM2NgUUwdOU_JYgCRDA5ACGjQfn30QlQsgKvcTVnkAovI-YZUPICpBvs0CAN1eY_c","id":"fresh","changes":[{"rev":"5-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-YjwiMtOdXCwJyU8ICYtABi0n6EnwzSzIxS00yI8hPEjAMQM-5nJTIQUPkAovI_UGUWAA0SgOI","id":"updated","changes":[{"rev":"2-7051cbe5c8faecd085a3fa619e6e6337"}]}
23824 {"seq":"3-g1AAAAHReJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D660H6ExlwqspjAZIMDUAKqHA-yCZGiEuTUy0MzEnxL8SkBRCT9iPcbJBmZpSaZkKUmyFmHICYcZ-wux9AVIJ8mAUABgp6XQ","id":"deleted","changes":[{"rev":"2-eec205a9d413992850a6e32678485900"}],"deleted":true}
23825 ... tum tee tum ...
23826 {"seq":"6-g1AAAAIreJyVkEsKwjAURWMrqCOXoCuQ9MU0OrI70XyppcaRY92J7kR3ojupaVNopRQsgRd4yTlwb44QmqahQnN7VjpKImAr7E6Uu4eAI7EoiiJLQx6c3GIiuVJcx93vvQqxdFPsaguqLAY04YwpNLtYpc3RatXPJyW__-EFllst4D_-UPLXmh9VPAaICaEDUtixm-jmLie6N30YqTeYDenDmx7e9GwyYRODNuu_MnnHyzverV6AMkPkAMfHO1rdUAKUkqhLZV-_0o5j","id":"updated","changes":[{"rev":"3-825cb35de44c433bfb2df415563a19de"}]}
23827 Obviously, … tum tee tum … does not appear in the actual response, but
23829 represents a long pause before the change with seq 6 occurred.
23830 Event Source
23832 The eventsource feed provides push notifications that can be consumed
23833 in the form of DOM events in the browser. Refer to the W3C eventsource
23834 specification for further details. CouchDB also honours the
23835 Last-Event-ID parameter.
23836 GET /somedatabase/_changes?feed=eventsource HTTP/1.1
23838 // define the event handling function
23840 if (window.EventSource) {
23841 var source = new EventSource("/somedatabase/_changes?feed=eventsource");
23843 source.onerror = function(e) {
23844 alert('EventSource failed.');
23845 };
23846 var results = [];
23848 var sourceListener = function(e) {
23849 var data = JSON.parse(e.data);
23850 results.push(data);
23851 };
23852 // start listening for events
23854 source.addEventListener('message', sourceListener, false);
23855 // stop listening for events
23857 source.removeEventListener('message', sourceListener, false);
23858 }
23860 If you set a heartbeat interval (using the heartbeat query argument),
23862 CouchDB will send a hearbeat event that you can subscribe to with:
23863 source.addEventListener('heartbeat', function () {}, false);
23865 This can be monitored by the client application to restart the
23867 EventSource connection if needed (i.e. if the TCP connection gets stuck
23868 in a half-open state).
23869 NOTE:
23871 EventSource connections are subject to cross-origin resource sharing
23872 restrictions. You might need to configure CORS support to get the
23873 EventSource to work in your application.
23874 Filtering
23876 You can filter the contents of the changes feed in a number of ways.
23877 The most basic way is to specify one or more document IDs to the query.
23878 This causes the returned structure value to only contain changes for
23879 the specified IDs. Note that the value of this query argument should be
23880 a JSON formatted array.
23881 You can also filter the _changes feed by defining a filter function
23883 within a design document. The specification for the filter is the same
23884 as for replication filters. You specify the name of the filter function
23885 to the filter parameter, specifying the design document name and filter
23886 name. For example:
23887 GET /db/_changes?filter=design_doc/filtername HTTP/1.1
23889 Additionally, a couple of built-in filters are available and described
23891 below.
23892 _doc_ids
23894 This filter accepts only changes for documents which ID in specified in
23895 doc_ids query parameter or payload’s object array. See POST
23896 /{db}/_changes for an example.
23897 _selector
23899 New in version 2.0.
23900 This filter accepts only changes for documents which match a specified
23903 selector, defined using the same selector syntax used for _find.
23904 This is significantly more efficient than using a JavaScript filter
23906 function and is the recommended option if filtering on document
23907 attributes only.
23908 Note that, unlike JavaScript filters, selectors do not have access to
23910 the request object.
23911 Request:
23913 POST /recipes/_changes?filter=_selector HTTP/1.1
23915 Content-Type: application/json
23916 Host: localhost:5984
23917 {
23919 "selector": { "_id": { "$regex": "^_design/" } }
23920 }
23921 Response:
23923 HTTP/1.1 200 OK
23925 Cache-Control: must-revalidate
23926 Content-Type: application/json
23927 Date: Tue, 06 Sep 2016 20:03:23 GMT
23928 Etag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
23929 Server: CouchDB (Erlang OTP/18)
23930 Transfer-Encoding: chunked
23931 {
23933 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
23934 "pending": 0,
23935 "results": [
23936 {
23937 "changes": [
23938 {
23939 "rev": "10-304cae84fd862832ea9814f02920d4b2"
23940 }
23941 ],
23942 "id": "_design/ingredients",
23943 "seq": "8-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-ZnJTIQULkAonI_ws0GaWZGqWkmRLkZYsYBiBn3Cdv2AKIS7ENWsG2mxkampsaGmLqyAOYpgEo"
23944 },
23945 {
23946 "changes": [
23947 {
23948 "rev": "123-6f7c1b7c97a9e4f0d22bdf130e8fd817"
23949 }
23950 ],
23951 "deleted": true,
23952 "id": "_design/cookbook",
23953 "seq": "9-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D661F8YWBkZGZsbEqCL_JYgCRDA5ACGjQ_K5GBgMoFEJX7EW42SDMzSk0zIcrNEDMOQMy4T9i2BxCVYB-ygm0zNTYyNTU2xNSVBQDnK4BL"
23954 },
23955 {
23956 "changes": [
23957 {
23958 "rev": "6-5b8a52c22580e922e792047cff3618f3"
23959 }
23960 ],
23961 "deleted": true,
23962 "id": "_design/meta",
23963 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
23964 }
23965 ]
23966 }
23967 Missing selector
23969 If the selector object is missing from the request body, the error mes‐
23970 sage is similar to the following example:
23971 {
23973 "error": "bad request",
23974 "reason": "Selector must be specified in POST payload"
23975 }
23976 Not a valid JSON object
23978 If the selector object is not a well-formed JSON object, the error mes‐
23979 sage is similar to the following example:
23980 {
23982 "error": "bad request",
23983 "reason": "Selector error: expected a JSON object"
23984 }
23985 Not a valid selector
23987 If the selector object does not contain a valid selection expression,
23988 the error message is similar to the following example:
23989 {
23991 "error": "bad request",
23992 "reason": "Selector error: expected a JSON object"
23993 }
23994 _design
23996 The _design filter accepts only changes for any design document within
23997 the requested database.
23998 Request:
24000 GET /recipes/_changes?filter=_design HTTP/1.1
24002 Accept: application/json
24003 Host: localhost:5984
24004 Response:
24006 HTTP/1.1 200 OK
24008 Cache-Control: must-revalidate
24009 Content-Type: application/json
24010 Date: Tue, 06 Sep 2016 12:55:12 GMT
24011 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
24012 Server: CouchDB (Erlang OTP)
24013 Transfer-Encoding: chunked
24014 {
24016 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
24017 "pending": 0,
24018 "results": [
24019 {
24020 "changes": [
24021 {
24022 "rev": "10-304cae84fd862832ea9814f02920d4b2"
24023 }
24024 ],
24025 "id": "_design/ingredients",
24026 "seq": "8-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D666H6GcH6DYyMzIyNTUnwRR4LkGRoAFJAg-ZnJTIQULkAonI_ws0GaWZGqWkmRLkZYsYBiBn3Cdv2AKIS7ENWsG2mxkampsaGmLqyAOYpgEo"
24027 },
24028 {
24029 "changes": [
24030 {
24031 "rev": "123-6f7c1b7c97a9e4f0d22bdf130e8fd817"
24032 }
24033 ],
24034 "deleted": true,
24035 "id": "_design/cookbook",
24036 "seq": "9-g1AAAAHxeJzLYWBg4MhgTmHgz8tPSTV0MDQy1zMAQsMcoARTIkOS_P___7MymBOZcoEC7MmJKSmJqWaYynEakaQAJJPsoaYwgE1JM0o1TjQ3T2HgLM1LSU3LzEtNwa3fAaQ_HkV_kkGyZWqSEXH6E0D661F8YWBkZGZsbEqCL_JYgCRDA5ACGjQ_K5GBgMoFEJX7EW42SDMzSk0zIcrNEDMOQMy4T9i2BxCVYB-ygm0zNTYyNTU2xNSVBQDnK4BL"
24037 },
24038 {
24039 "changes": [
24040 {
24041 "rev": "6-5b8a52c22580e922e792047cff3618f3"
24042 }
24043 ],
24044 "deleted": true,
24045 "id": "_design/meta",
24046 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
24047 }
24048 ]
24049 }
24050 _view
24052 New in version 1.2.
24053 The special filter _view allows to use existing map function as the
24056 filter. If the map function emits anything for the processed document
24057 it counts as accepted and the changes event emits to the feed. For most
24058 use-practice cases filter functions are very similar to map ones, so
24059 this feature helps to reduce amount of duplicated code.
24060 WARNING:
24062 While map functions doesn’t process the design documents, using
24063 _view filter forces them to do this. You need to be sure, that they
24064 are ready to handle documents with alien structure without panic.
24065 NOTE:
24067 Using _view filter doesn’t queries the view index files, so you can‐
24068 not use common view query parameters to additionally filter the
24069 changes feed by index key. Also, CouchDB doesn’t returns the result
24070 instantly as it does for views - it really uses the specified map
24071 function as filter.
24072 Moreover, you cannot make such filters dynamic e.g. process the
24074 request query parameters or handle the userctx_object - the map
24075 function is only operates with the document.
24076 Request:
24078 GET /recipes/_changes?filter=_view&view=ingredients/by_recipe HTTP/1.1
24080 Accept: application/json
24081 Host: localhost:5984
24082 Response:
24084 HTTP/1.1 200 OK
24086 Cache-Control: must-revalidate
24087 Content-Type: application/json
24088 Date: Tue, 06 Sep 2016 12:57:56 GMT
24089 ETag: "ARIHFWL3I7PIS0SPVTFU6TLR2"
24090 Server: CouchDB (Erlang OTP)
24091 Transfer-Encoding: chunked
24092 {
24094 "last_seq": "11-g1AAAAIreJyVkEEKwjAQRUOrqCuPoCeQZGIaXdmbaNIk1FLjyrXeRG-iN9Gb1LQRaimFlsAEJnkP_s8RQtM0VGhuz0qTmABfYXdI7h4CgeSiKIosDUVwcotJIpQSOmp_71TIpZty97OgymJAU8G5QrOLVdocrVbdfFzy-wYvcbLVEvrxh5K_NlJggIhSNiCFHbmJbu5yonttMoneYD6kD296eNOzzoRNBNqse2Xyjpd3vP96AcYNTQY4Pt5RdTOuHIwCY5S0qewLwY6OaA",
24095 "results": [
24096 {
24097 "changes": [
24098 {
24099 "rev": "13-bcb9d6388b60fd1e960d9ec4e8e3f29e"
24100 }
24101 ],
24102 "id": "SpaghettiWithMeatballs",
24103 "seq": "11-g1AAAAIReJyVkE0OgjAQRiegUVceQU9g-mOpruQm2tI2SLCuXOtN9CZ6E70JFmpCCCFCmkyTdt6bfJMDwDQNFcztWWkcY8JXyB2cu49AgFwURZGloQhO7mGSCKWEjtrtnQq5dFXufhaoLIZoKjhXMLtYpc3RatXNxyW_b_ASJVstST_-UPLXRgpESEQpG5DCjlyFm7uc6F6bTKI3iA_Zhzc9vOlZZ0ImItqse2Xyjpd3vDMBfzo_vrPawLiaxihhjOI2lX0BirqHbg"
24104 }
24105 ]
24106 }
24107 /db/_compact
24109 POST /{db}/_compact
24110 Request compaction of the specified database. Compaction com‐
24111 presses the disk database file by performing the following oper‐
24112 ations:
24113 · Writes a new, optimised, version of the database file, remov‐
24115 ing any unused sections from the new version during write.
24116 Because a new file is temporarily created for this purpose,
24117 you may require up to twice the current storage space of the
24118 specified database in order for the compaction routine to com‐
24119 plete.
24120 · Removes the bodies of any non-leaf revisions of documents from
24122 the database.
24123 · Removes old revision history beyond the limit specified by the
24125 _revs_limit database parameter.
24126 Compaction can only be requested on an individual database; you
24128 cannot compact all the databases for a CouchDB instance. The
24129 compaction process runs as a background process.
24130 You can determine if the compaction process is operating on a
24132 database by obtaining the database meta information, the com‐
24133 pact_running value of the returned database structure will be
24134 set to true. See GET /{db}.
24135 You can also obtain a list of running processes to determine
24137 whether compaction is currently running. See
24138 api/server/active_tasks.
24139 Parameters
24141 · db – Database name
24143 Request Headers
24145 · Accept – .INDENT 2.0
24147 · application/json
24149 · text/plain
24151 · Content-Type – application/json
24154 Response Headers
24156 · Content-Type – .INDENT 2.0
24158 · application/json
24160 · text/plain; charset=utf-8
24162 Response JSON Object
24165 · ok (boolean) – Operation status
24167 Status Codes
24169 · 202 Accepted – Compaction request has been accepted
24171 · 400 Bad Request – Invalid database name
24173 · 401 Unauthorized – CouchDB Server Administrator privileges
24175 required
24176 · 415 Unsupported Media Type – Bad Content-Type value
24178
24180 POST /db/_compact HTTP/1.1
24182 Accept: application/json
24183 Content-Type: application/json
24184 Host: localhost:5984
24185 Response:
24187 HTTP/1.1 202 Accepted
24189 Cache-Control: must-revalidate
24190 Content-Length: 12
24191 Content-Type: application/json
24192 Date: Mon, 12 Aug 2013 09:27:43 GMT
24193 Server: CouchDB (Erlang/OTP)
24194 {
24196 "ok": true
24197 }
24198 /db/_compact/design-doc
24200 POST /{db}/_compact/{ddoc}
24201 Compacts the view indexes associated with the specified design
24202 document. It may be that compacting a large view can return
24203 more storage than compacting the actual db. Thus, you can use
24204 this in place of the full database compaction if you know a spe‐
24205 cific set of view indexes have been affected by a recent data‐
24206 base change.
24207 Parameters
24209 · db – Database name
24211 · ddoc – Design document name
24213 Request Headers
24215 · Accept – .INDENT 2.0
24217 · application/json
24219 · text/plain
24221 · Content-Type – application/json
24224 Response Headers
24226 · Content-Type – .INDENT 2.0
24228 · application/json
24230 · text/plain; charset=utf-8
24232 Response JSON Object
24235 · ok (boolean) – Operation status
24237 Status Codes
24239 · 202 Accepted – Compaction request has been accepted
24241 · 400 Bad Request – Invalid database name
24243 · 401 Unauthorized – CouchDB Server Administrator privileges
24245 required
24246 · 404 Not Found – Design document not found
24248 · 415 Unsupported Media Type – Bad Content-Type value
24250
24252 POST /db/_compact/posts HTTP/1.1
24254 Accept: application/json
24255 Content-Type: application/json
24256 Host: localhost:5984
24257 Response:
24259 HTTP/1.1 202 Accepted
24261 Cache-Control: must-revalidate
24262 Content-Length: 12
24263 Content-Type: application/json
24264 Date: Mon, 12 Aug 2013 09:36:44 GMT
24265 Server: CouchDB (Erlang/OTP)
24266 {
24268 "ok": true
24269 }
24270 NOTE:
24272 View indexes are stored in a separate .couch file based on a
24273 hash of the design document’s relevant functions, in a sub
24274 directory of where the main .couch database files are
24275 located.
24276 /db/_ensure_full_commit
24278 POST /{db}/_ensure_full_commit
24279 Changed in version 3.0.0: Deprecated; endpoint is a no-op.
24280 Before 3.0 this was used to commit recent changes to the data‐
24283 base in case the delayed_commits=true option was set. That
24284 option is always false now, so commits are never delayed. How‐
24285 ever, this endpoint is kept for compatibility with older repli‐
24286 cators.
24287 Parameters
24289 · db – Database name
24291 Request Headers
24293 · Accept – .INDENT 2.0
24295 · application/json
24297 · text/plain
24299 · Content-Type – application/json
24302 Response Headers
24304 · Content-Type – .INDENT 2.0
24306 · application/json
24308 · text/plain; charset=utf-8
24310 Response JSON Object
24313 · instance_start_time (string) – Always "0". (Returned for
24315 legacy reasons.)
24316 · ok (boolean) – Operation status
24318 Status Codes
24320 · 201 Created – Commit completed successfully
24322 · 400 Bad Request – Invalid database name
24324 · 415 Unsupported Media Type – Bad Content-Type value
24326
24328 POST /db/_ensure_full_commit HTTP/1.1
24330 Accept: application/json
24331 Content-Type: application/json
24332 Host: localhost:5984
24333 Response:
24335 HTTP/1.1 201 Created
24337 Cache-Control: must-revalidate
24338 Content-Length: 53
24339 Content-Type: application/json
24340 Date: Mon, 12 Aug 2013 10:22:19 GMT
24341 Server: CouchDB (Erlang/OTP)
24342 {
24344 "instance_start_time": "0",
24345 "ok": true
24346 }
24347 /db/_view_cleanup
24349 POST /{db}/_view_cleanup
24350 Removes view index files that are no longer required by CouchDB
24351 as a result of changed views within design documents. As the
24352 view filename is based on a hash of the view functions, over
24353 time old views will remain, consuming storage. This call cleans
24354 up the cached view output on disk for a given view.
24355 Parameters
24357 · db – Database name
24359 Request Headers
24361 · Accept – .INDENT 2.0
24363 · application/json
24365 · text/plain
24367 · Content-Type – application/json
24370 Response Headers
24372 · Content-Type – .INDENT 2.0
24374 · application/json
24376 · text/plain; charset=utf-8
24378 Response JSON Object
24381 · ok (boolean) – Operation status
24383 Status Codes
24385 · 202 Accepted – Compaction request has been accepted
24387 · 400 Bad Request – Invalid database name
24389 · 401 Unauthorized – CouchDB Server Administrator privileges
24391 required
24392 · 415 Unsupported Media Type – Bad Content-Type value
24394
24396 POST /db/_view_cleanup HTTP/1.1
24398 Accept: application/json
24399 Content-Type: application/json
24400 Host: localhost:5984
24401 Response:
24403 HTTP/1.1 202 Accepted
24405 Cache-Control: must-revalidate
24406 Content-Length: 12
24407 Content-Type: application/json
24408 Date: Mon, 12 Aug 2013 09:27:43 GMT
24409 Server: CouchDB (Erlang/OTP)
24410 {
24412 "ok": true
24413 }
24414 /db/_security
24416 GET /{db}/_security
24417 Returns the current security object from the specified database.
24418 The security object consists of two compulsory elements, admins
24420 and members, which are used to specify the list of users and/or
24421 roles that have admin and members rights to the database respec‐
24422 tively:
24423 · members: they can read all types of documents from the DB, and
24425 they can write (and edit) documents to the DB except for
24426 design documents.
24427 · admins: they have all the privileges of members plus the priv‐
24429 ileges: write (and edit) design documents, add/remove database
24430 admins and members and set the database revisions limit. They
24431 can not create a database nor delete a database.
24432 Both members and admins objects contain two array-typed fields:
24434 · names: List of CouchDB user names
24436 · roles: List of users roles
24438 Any additional fields in the security object are optional. The
24440 entire security object is made available to validation and other
24441 internal functions so that the database can control and limit
24442 functionality.
24443 If both the names and roles fields of either the admins or mem‐
24445 bers properties are empty arrays, or are not existent, it means
24446 the database has no admins or members.
24447 Having no admins, only server admins (with the reserved _admin
24449 role) are able to update design document and make other admin
24450 level changes.
24451 Having no members, any user can write regular documents (any
24453 non-design document) and read documents from the database.
24454 If there are any member names or roles defined for a database,
24456 then only authenticated users having a matching name or role are
24457 allowed to read documents from the database (or do a GET /{db}
24458 call).
24459 NOTE:
24461 If the security object for a database has never been set,
24462 then the value returned will be empty.
24463 Also note, that security objects are not regular versioned
24465 documents (that is, they are not under MVCC rules). This is a
24466 design choice to speed up authorization checks (avoids
24467 traversing a database’s documents B-Tree).
24468 Parameters
24470 · db – Database name
24472 Request Headers
24474 · Accept – .INDENT 2.0
24476 · application/json
24478 · text/plain
24480 Response Headers
24483 · Content-Type – .INDENT 2.0
24485 · application/json
24487 · text/plain; charset=utf-8
24489 Response JSON Object
24492 · admins (object) – Object with two fields as names and roles.
24494 See description above for more info.
24495 · members (object) – Object with two fields as names and roles.
24497 See description above for more info.
24498 Status Codes
24500 · 200 OK – Request completed successfully
24502
24504 GET /db/_security HTTP/1.1
24506 Accept: application/json
24507 Host: localhost:5984
24508 Response:
24510 HTTP/1.1 200 OK
24512 Cache-Control: must-revalidate
24513 Content-Length: 109
24514 Content-Type: application/json
24515 Date: Mon, 12 Aug 2013 19:05:29 GMT
24516 Server: CouchDB (Erlang/OTP)
24517 {
24519 "admins": {
24520 "names": [
24521 "superuser"
24522 ],
24523 "roles": [
24524 "admins"
24525 ]
24526 },
24527 "members": {
24528 "names": [
24529 "user1",
24530 "user2"
24531 ],
24532 "roles": [
24533 "developers"
24534 ]
24535 }
24536 }
24537 PUT /{db}/_security
24539 Sets the security object for the given database.
24540 Parameters
24542 · db – Database name
24544 Request Headers
24546 · Accept – .INDENT 2.0
24548 · application/json
24550 · text/plain
24552 · Content-Type – application/json
24555 Request JSON Object
24557 · admins (object) – Object with two fields as names and roles.
24559 See description above for more info.
24560 · members (object) – Object with two fields as names and roles.
24562 See description above for more info.
24563 Response Headers
24565 · Content-Type – .INDENT 2.0
24567 · application/json
24569 · text/plain; charset=utf-8
24571 Response JSON Object
24574 · ok (boolean) – Operation status
24576 Status Codes
24578 · 200 OK – Request completed successfully
24580 · 401 Unauthorized – CouchDB Server Administrator privileges
24582 required
24583
24585 shell> curl http://localhost:5984/pineapple/_security -X PUT -H 'content-type: application/json' -H 'accept: application/json' -d '{"admins":{"names":["superuser"],"roles":["admins"]},"members":{"names": ["user1","user2"],"roles": ["developers"]}}'
24587 PUT /db/_security HTTP/1.1
24589 Accept: application/json
24590 Content-Length: 121
24591 Content-Type: application/json
24592 Host: localhost:5984
24593 {
24595 "admins": {
24596 "names": [
24597 "superuser"
24598 ],
24599 "roles": [
24600 "admins"
24601 ]
24602 },
24603 "members": {
24604 "names": [
24605 "user1",
24606 "user2"
24607 ],
24608 "roles": [
24609 "developers"
24610 ]
24611 }
24612 }
24613 Response:
24615 HTTP/1.1 200 OK
24617 Cache-Control: must-revalidate
24618 Content-Length: 12
24619 Content-Type: application/json
24620 Date: Tue, 13 Aug 2013 11:26:28 GMT
24621 Server: CouchDB (Erlang/OTP)
24622 {
24624 "ok": true
24625 }
24626 /db/_purge
24628 POST /{db}/_purge
24629 A database purge permanently removes the references to documents
24630 in the database. Normal deletion of a document within CouchDB
24631 does not remove the document from the database, instead, the
24632 document is marked as _deleted=true (and a new revision is cre‐
24633 ated). This is to ensure that deleted documents can be repli‐
24634 cated to other databases as having been deleted. This also means
24635 that you can check the status of a document and identify that
24636 the document has been deleted by its absence.
24637 The purge request must include the document IDs, and for each
24639 document ID, one or more revisions that must be purged. Docu‐
24640 ments can be previously deleted, but it is not necessary. Revi‐
24641 sions must be leaf revisions.
24642 The response will contain a list of the document IDs and revi‐
24644 sions successfully purged.
24645 Parameters
24647 · db – Database name
24649 Request Headers
24651 · Accept – .INDENT 2.0
24653 · application/json
24655 · text/plain
24657 · Content-Type – application/json
24660 Request JSON Object
24662 · object – Mapping of document ID to list of revisions to purge
24664 Response Headers
24666 · Content-Type – .INDENT 2.0
24668 · application/json
24670 · text/plain; charset=utf-8
24672 Response JSON Object
24675 · purge_seq (string) – Purge sequence string
24677 · purged (object) – Mapping of document ID to list of purged
24679 revisions
24680 Status Codes
24682 · 201 Created – Request completed successfully
24684 · 202 Accepted – Request was accepted, and was completed suc‐
24686 cessfully on at least one replica, but quorum was not reached.
24687 · 400 Bad Request – Invalid database name or JSON payload
24689 · 415 Unsupported Media Type – Bad Content-Type value
24691 · 500 Internal Server Error – Internal server error or timeout
24693
24695 POST /db/_purge HTTP/1.1
24697 Accept: application/json
24698 Content-Length: 76
24699 Content-Type: application/json
24700 Host: localhost:5984
24701 {
24703 "c6114c65e295552ab1019e2b046b10e": [
24704 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b",
24705 "3-c50a32451890a3f1c3e423334cc92745"
24706 ]
24707 }
24708 Response:
24710 HTTP/1.1 201 Created
24712 Cache-Control: must-revalidate
24713 Content-Length: 107
24714 Content-Type: application/json
24715 Date: Fri, 02 Jun 2017 18:55:54 GMT
24716 Server: CouchDB/2.0.0-2ccd4bf (Erlang OTP/18)
24717 {
24719 "purge_seq": null,
24720 "purged": {
24721 "c6114c65e295552ab1019e2b046b10e": {
24722 "purged": [
24723 "3-c50a32451890a3f1c3e423334cc92745"
24724 ]
24725 }
24726 }
24727 }
24728 [image: Document Revision Tree 1] [image] Document Revision Tree
24729 1.UNINDENT
24730 For example, given the above purge tree and issuing the above purge
24732 request, the whole document will be purged, as it contains only a
24733 single branch with a leaf revision
24734 3-c50a32451890a3f1c3e423334cc92745 that will be purged. As a result
24735 of this purge operation, a document with
24736 _id:c6114c65e295552ab1019e2b046b10e will be completely removed from
24737 the database’s document b+tree, and sequence b+tree. It will not be
24738 available through _all_docs or _changes endpoints, as though this
24739 document never existed. Also as a result of purge operation, the
24740 database’s purge_seq and update_seq will be increased.
24741 Notice, how revision 3-b06fcd1c1c9e0ec7c480ee8aa467bf3b was ignored.
24743 Revisions that have already been purged and non-leaf revisions are
24744 ignored in a purge request.
24745 If a document has two conflict revisions with the following revision
24747 history:
24748 [image: Document Revision Tree 1] [image] Document Revision Tree
24749 2.UNINDENT
24750 the above purge request will purge only one branch, leaving the docu‐
24752 ment’s revision tree with only a single branch:
24753 [image: Document Revision Tree 3] [image] Document Revision Tree
24754 3.UNINDENT
24755 As a result of this purge operation, a new updated version of the
24757 document will be available in _all_docs and _changes, creating a new
24758 record in _changes. The database’s purge_seq and update_seq will be
24759 increased.
24760 Internal Replication
24762 Purges are automatically replicated between replicas of the same data‐
24763 base. Each database has an internal purge tree that stores a certain
24764 number of the most recent purges. This allows internal synchonization
24765 between replicas of the same database.
24766 External Replication
24768 Purge operations are not replicated to other external databases. Exter‐
24769 nal replication works by identifying a source’s document revisions that
24770 are missing on target, and copying these revisions from source to tar‐
24771 get. A purge operation completely purges revisions from a document’s
24772 purge tree making external replication of purges impossible.
24773 NOTE:
24775 If you need a purge to be effective across multiple effective
24776 databases, you must run the purge separately on each of the
24777 databases.
24778 Updating Indexes
24780 The number of purges on a database is tracked using a purge sequence.
24781 This is used by the view indexer to optimize the updating of views that
24782 contain the purged documents.
24783 Each internal database indexer, including the view indexer, keeps its
24785 own purge sequence. The purge sequence stored in the index can be much
24786 smaller than the database’s purge sequence up to the number of purge
24787 requests allowed to be stored in the purge trees of the database. Mul‐
24788 tiple purge requests can be processed by the indexer without incurring
24789 a rebuild of the index. The index will be updated according to these
24790 purge requests.
24791 The index of documents is based on the winner of the revision tree.
24793 Depending on which revision is specified in the purge request, the
24794 index update observes the following behavior:
24795 · If the winner of the revision tree is not specified in the purge
24797 request, there is no change to the index record of this document.
24798 · If the winner of the revision tree is specified in the purge request,
24800 and there is still a revision left after purging, the index record of
24801 the document will be built according to the new winner of the revi‐
24802 sion tree.
24803 · If all revisions of the document are specified in the purge request,
24805 the index record of the document will be deleted. The document will
24806 no longer be found in searches.
24807 /db/_purged_infos_limit
24809 GET /{db}/_purged_infos_limit
24810 Gets the current purged_infos_limit (purged documents limit)
24811 setting, the maximum number of historical purges (purged docu‐
24812 ment Ids with their revisions) that can be stored in the data‐
24813 base.
24814 Parameters
24816 · db – Database name
24818 Request Headers
24820 · Accept – .INDENT 2.0
24822 · application/json
24824 · text/plain
24826 Response Headers
24829 · Content-Type – .INDENT 2.0
24831 · application/json
24833 · text/plain; charset=utf-8
24835 Status Codes
24838 · 200 OK – Request completed successfully
24840 Request:
24842 GET /db/_purged_infos_limit HTTP/1.1
24844 Accept: application/json
24845 Host: localhost:5984
24846 Response:
24848 HTTP/1.1 200 OK
24850 Cache-Control: must-revalidate
24851 Content-Length: 5
24852 Content-Type: application/json
24853 Date: Wed, 14 Jun 2017 14:43:42 GMT
24854 Server: CouchDB (Erlang/OTP)
24855 1000
24857 PUT /{db}/_purged_infos_limit
24859 Sets the maximum number of purges (requested purged Ids with
24860 their revisions) that will be tracked in the database, even
24861 after compaction has occurred. You can set the purged documents
24862 limit on a database with a scalar integer of the limit that you
24863 want to set as the request body.
24864 The default value of historical stored purges is 1000. This
24866 means up to 1000 purges can be synchronized between replicas of
24867 the same databases in case of one of the replicas was down when
24868 purges occurred.
24869 This request sets the soft limit for stored purges. During the
24871 compaction CouchDB will try to keep only _purged_infos_limit of
24872 purges in the database, but occasionally the number of stored
24873 purges can exceed this value. If a database has not completed
24874 purge synchronization with active indexes or active internal
24875 replications, it may temporarily store a higher number of his‐
24876 torical purges.
24877 Parameters
24879 · db – Database name
24881 Request Headers
24883 · Accept – .INDENT 2.0
24885 · application/json
24887 · text/plain
24889 · Content-Type – application/json
24892 Response Headers
24894 · Content-Type – .INDENT 2.0
24896 · application/json
24898 · text/plain; charset=utf-8
24900 Response JSON Object
24903 · ok (boolean) – Operation status
24905 Status Codes
24907 · 200 OK – Request completed successfully
24909 · 400 Bad Request – Invalid JSON data
24911 Request:
24913 PUT /db/_purged_infos_limit HTTP/1.1
24915 Accept: application/json
24916 Content-Length: 4
24917 Content-Type: application/json
24918 Host: localhost:5984
24919 1500
24921 Response:
24923 HTTP/1.1 200 OK
24925 Cache-Control: must-revalidate
24926 Content-Length: 12
24927 Content-Type: application/json
24928 Date: Wed, 14 Jun 2017 14:45:34 GMT
24929 Server: CouchDB (Erlang/OTP)
24930 {
24932 "ok": true
24933 }
24934 /db/_missing_revs
24936 POST /{db}/_missing_revs
24937 With given a list of document revisions, returns the document
24938 revisions that do not exist in the database.
24939 Parameters
24941 · db – Database name
24943 Request Headers
24945 · Accept – .INDENT 2.0
24947 · application/json
24949 · text/plain
24951 · Content-Type – application/json
24954 Request JSON Object
24956 · object – Mapping of document ID to list of revisions to lookup
24958 Response Headers
24960 · Content-Type – .INDENT 2.0
24962 · application/json
24964 · text/plain; charset=utf-8
24966 Response JSON Object
24969 · missing_revs (object) – Mapping of document ID to list of
24971 missed revisions
24972 Status Codes
24974 · 200 OK – Request completed successfully
24976 · 400 Bad Request – Invalid database name or JSON payload
24978
24980 POST /db/_missing_revs HTTP/1.1
24982 Accept: application/json
24983 Content-Length: 76
24984 Content-Type: application/json
24985 Host: localhost:5984
24986 {
24988 "c6114c65e295552ab1019e2b046b10e": [
24989 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b",
24990 "3-0e871ef78849b0c206091f1a7af6ec41"
24991 ]
24992 }
24993 Response:
24995 HTTP/1.1 200 OK
24997 Cache-Control: must-revalidate
24998 Content-Length: 64
24999 Content-Type: application/json
25000 Date: Mon, 12 Aug 2013 10:53:24 GMT
25001 Server: CouchDB (Erlang/OTP)
25002 {
25004 "missed_revs":{
25005 "c6114c65e295552ab1019e2b046b10e": [
25006 "3-b06fcd1c1c9e0ec7c480ee8aa467bf3b"
25007 ]
25008 }
25009 }
25010 /db/_revs_diff
25012 POST /{db}/_revs_diff
25013 Given a set of document/revision IDs, returns the subset of
25014 those that do not correspond to revisions stored in the data‐
25015 base.
25016 Its primary use is by the replicator, as an important optimiza‐
25018 tion: after receiving a set of new revision IDs from the source
25019 database, the replicator sends this set to the destination data‐
25020 base’s _revs_diff to find out which of them already exist there.
25021 It can then avoid fetching and sending already-known document
25022 bodies.
25023 Both the request and response bodies are JSON objects whose keys
25025 are document IDs; but the values are structured differently:
25026 · In the request, a value is an array of revision IDs for that
25028 document.
25029 · In the response, a value is an object with a missing: key,
25031 whose value is a list of revision IDs for that document (the
25032 ones that are not stored in the database) and optionally a
25033 possible_ancestors key, whose value is an array of revision
25034 IDs that are known that might be ancestors of the missing
25035 revisions.
25036 Parameters
25038 · db – Database name
25040 Request Headers
25042 · Accept – .INDENT 2.0
25044 · application/json
25046 · text/plain
25048 · Content-Type – application/json
25051 Request JSON Object
25053 · object – Mapping of document ID to list of revisions to lookup
25055 Response Headers
25057 · Content-Type – .INDENT 2.0
25059 · application/json
25061 · text/plain; charset=utf-8
25063 Response JSON Object
25066 · missing (array) – List of missed revisions for specified docu‐
25068 ment
25069 · possible_ancestors (array) – List of revisions that may be
25071 ancestors for specified document and its current revision in
25072 requested database
25073 Status Codes
25075 · 200 OK – Request completed successfully
25077 · 400 Bad Request – Invalid database name or JSON payload
25079
25081 POST /db/_revs_diff HTTP/1.1
25083 Accept: application/json
25084 Content-Length: 113
25085 Content-Type: application/json
25086 Host: localhost:5984
25087 {
25089 "190f721ca3411be7aa9477db5f948bbb": [
25090 "3-bb72a7682290f94a985f7afac8b27137",
25091 "4-10265e5a26d807a3cfa459cf1a82ef2e",
25092 "5-067a00dff5e02add41819138abb3284d"
25093 ]
25094 }
25095 Response:
25097 HTTP/1.1 200 OK
25099 Cache-Control: must-revalidate
25100 Content-Length: 88
25101 Content-Type: application/json
25102 Date: Mon, 12 Aug 2013 16:56:02 GMT
25103 Server: CouchDB (Erlang/OTP)
25104 {
25106 "190f721ca3411be7aa9477db5f948bbb": {
25107 "missing": [
25108 "3-bb72a7682290f94a985f7afac8b27137",
25109 "5-067a00dff5e02add41819138abb3284d"
25110 ],
25111 "possible_ancestors": [
25112 "4-10265e5a26d807a3cfa459cf1a82ef2e"
25113 ]
25114 }
25115 }
25116 /db/_revs_limit
25118 GET /{db}/_revs_limit
25119 Gets the current revs_limit (revision limit) setting.
25120 Parameters
25122 · db – Database name
25124 Request Headers
25126 · Accept – .INDENT 2.0
25128 · application/json
25130 · text/plain
25132 Response Headers
25135 · Content-Type – .INDENT 2.0
25137 · application/json
25139 · text/plain; charset=utf-8
25141 Status Codes
25144 · 200 OK – Request completed successfully
25146
25148 GET /db/_revs_limit HTTP/1.1
25150 Accept: application/json
25151 Host: localhost:5984
25152 Response:
25154 HTTP/1.1 200 OK
25156 Cache-Control: must-revalidate
25157 Content-Length: 5
25158 Content-Type: application/json
25159 Date: Mon, 12 Aug 2013 17:27:30 GMT
25160 Server: CouchDB (Erlang/OTP)
25161 1000
25163 PUT /{db}/_revs_limit
25165 Sets the maximum number of document revisions that will be
25166 tracked by CouchDB, even after compaction has occurred. You can
25167 set the revision limit on a database with a scalar integer of
25168 the limit that you want to set as the request body.
25169 Parameters
25171 · db – Database name
25173 Request Headers
25175 · Accept – .INDENT 2.0
25177 · application/json
25179 · text/plain
25181 · Content-Type – application/json
25184 Response Headers
25186 · Content-Type – .INDENT 2.0
25188 · application/json
25190 · text/plain; charset=utf-8
25192 Response JSON Object
25195 · ok (boolean) – Operation status
25197 Status Codes
25199 · 200 OK – Request completed successfully
25201 · 400 Bad Request – Invalid JSON data
25203
25205 PUT /db/_revs_limit HTTP/1.1
25207 Accept: application/json
25208 Content-Length: 5
25209 Content-Type: application/json
25210 Host: localhost:5984
25211 1000
25213 Response:
25215 HTTP/1.1 200 OK
25217 Cache-Control: must-revalidate
25218 Content-Length: 12
25219 Content-Type: application/json
25220 Date: Mon, 12 Aug 2013 17:47:52 GMT
25221 Server: CouchDB (Erlang/OTP)
25222 {
25224 "ok": true
25225 }
25226 Documents
25228 Details on how to create, read, update and delete documents within a
25229 database.
25230 /db/doc
25232 HEAD /{db}/{docid}
25233 Returns the HTTP Headers containing a minimal amount of informa‐
25234 tion about the specified document. The method supports the same
25235 query arguments as the GET /{db}/{docid} method, but only the
25236 header information (including document size, and the revision as
25237 an ETag), is returned.
25238 The ETag header shows the current revision for the requested
25240 document, and the Content-Length specifies the length of the
25241 data, if the document were requested in full.
25242 Adding any of the query arguments (see GET /{db}/{docid}), then
25244 the resulting HTTP Headers will correspond to what would be
25245 returned.
25246 Parameters
25248 · db – Database name
25250 · docid – Document ID
25252 Request Headers
25254 · If-None-Match – Double quoted document’s revision token
25256 Response Headers
25258 · Content-Length – Document size
25260 · ETag – Double quoted document’s revision token
25262 Status Codes
25264 · 200 OK – Document exists
25266 · 304 Not Modified – Document wasn’t modified since spec‐
25268 ified revision
25269 · 401 Unauthorized – Read privilege required
25271 · 404 Not Found – Document not found
25273 Request:
25275 HEAD /db/SpaghettiWithMeatballs HTTP/1.1
25277 Accept: application/json
25278 Host: localhost:5984
25279 Response:
25281 HTTP/1.1 200 OK
25283 Cache-Control: must-revalidate
25284 Content-Length: 660
25285 Content-Type: application/json
25286 Date: Tue, 13 Aug 2013 21:35:37 GMT
25287 ETag: "12-151bb8678d45aaa949ec3698ef1c7e78"
25288 Server: CouchDB (Erlang/OTP)
25289 GET /{db}/{docid}
25291 Returns document by the specified docid from the specified db.
25292 Unless you request a specific revision, the latest revision of
25293 the document will always be returned.
25294 Parameters
25296 · db – Database name
25298 · docid – Document ID
25300 Request Headers
25302 · Accept – .INDENT 2.0
25304 · application/json
25306 · multipart/related
25308 · multipart/mixed
25310 · text/plain
25312 · If-None-Match – Double quoted document’s revision token
25315 Query Parameters
25317 · attachments (boolean) – Includes attachments bodies in
25319 response. Default is false
25320 · att_encoding_info (boolean) – Includes encoding information in
25322 attachment stubs if the particular attachment is compressed.
25323 Default is false.
25324 · atts_since (array) – Includes attachments only since specified
25326 revisions. Doesn’t includes attachments for specified revi‐
25327 sions. Optional
25328 · conflicts (boolean) – Includes information about conflicts in
25330 document. Default is false
25331 · deleted_conflicts (boolean) – Includes information about
25333 deleted conflicted revisions. Default is false
25334 · latest (boolean) – Forces retrieving latest “leaf” revision,
25336 no matter what rev was requested. Default is false
25337 · local_seq (boolean) – Includes last update sequence for the
25339 document. Default is false
25340 · meta (boolean) – Acts same as specifying all conflicts,
25342 deleted_conflicts and revs_info query parameters. Default is
25343 false
25344 · open_revs (array) – Retrieves documents of specified leaf
25346 revisions. Additionally, it accepts value as all to return
25347 all leaf revisions. Optional
25348 · rev (string) – Retrieves document of specified revision.
25350 Optional
25351 · revs (boolean) – Includes list of all known document revi‐
25353 sions. Default is false
25354 · revs_info (boolean) – Includes detailed information for all
25356 known document revisions. Default is false
25357 Response Headers
25359 · Content-Type – .INDENT 2.0
25361 · application/json
25363 · multipart/related
25365 · multipart/mixed
25367 · text/plain; charset=utf-8
25369 · ETag – Double quoted document’s revision token. Not available when
25372 retrieving conflicts-related information
25373 · Transfer-Encoding – chunked. Available if requested with query param‐
25375 eter open_revs
25376 Response JSON Object
25378 · _id (string) – Document ID
25380 · _rev (string) – Revision MVCC token
25382 · _deleted (boolean) – Deletion flag. Available if document was
25384 removed
25385 · _attachments (object) – Attachment’s stubs. Available if docu‐
25387 ment has any attachments
25388 · _conflicts (array) – List of conflicted revisions. Available
25390 if requested with conflicts=true query parameter
25391 · _deleted_conflicts (array) – List of deleted conflicted revi‐
25393 sions. Available if requested with deleted_conflicts=true
25394 query parameter
25395 · _local_seq (string) – Document’s update sequence in current
25397 database. Available if requested with local_seq=true query
25398 parameter
25399 · _revs_info (array) – List of objects with information about
25401 local revisions and their status. Available if requested with
25402 open_revs query parameter
25403 · _revisions (object) – List of local revision tokens without.
25405 Available if requested with revs=true query parameter
25406 Status Codes
25408 · 200 OK – Request completed successfully
25410 · 304 Not Modified – Document wasn’t modified since specified
25412 revision
25413 · 400 Bad Request – The format of the request or revision was
25415 invalid
25416 · 401 Unauthorized – Read privilege required
25418 · 404 Not Found – Document not found
25420
25422 GET /recipes/SpaghettiWithMeatballs HTTP/1.1
25424 Accept: application/json
25425 Host: localhost:5984
25426 Response:
25428 HTTP/1.1 200 OK
25430 Cache-Control: must-revalidate
25431 Content-Length: 660
25432 Content-Type: application/json
25433 Date: Tue, 13 Aug 2013 21:35:37 GMT
25434 ETag: "1-917fa2381192822767f010b95b45325b"
25435 Server: CouchDB (Erlang/OTP)
25436 {
25438 "_id": "SpaghettiWithMeatballs",
25439 "_rev": "1-917fa2381192822767f010b95b45325b",
25440 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25441 "ingredients": [
25442 "spaghetti",
25443 "tomato sauce",
25444 "meatballs"
25445 ],
25446 "name": "Spaghetti with meatballs"
25447 }
25448 PUT /{db}/{docid}
25450 The PUT method creates a new named document, or creates a new
25451 revision of the existing document. Unlike the POST /{db}, you
25452 must specify the document ID in the request URL.
25453 When updating an existing document, the current document revi‐
25455 sion must be included in the document (i.e. the request body),
25456 as the rev query parameter, or in the If-Match request header.
25457 Parameters
25459 · db – Database name
25461 · docid – Document ID
25463 Request Headers
25465 · Accept – .INDENT 2.0
25467 · application/json
25469 · text/plain
25471 · Content-Type – .INDENT 2.0
25474 · application/json
25476 · multipart/related
25478 · If-Match – Document’s revision. Alternative to rev query parameter or
25481 document key. Optional
25482 Query Parameters
25484 · rev (string) – Document’s revision if updating an existing
25486 document. Alternative to If-Match header or document key.
25487 Optional
25488 · batch (string) – Stores document in batch mode. Possible val‐
25490 ues: ok. Optional
25491 · new_edits (boolean) – Prevents insertion of a conflicting doc‐
25493 ument. Possible values: true (default) and false. If false, a
25494 well-formed _rev must be included in the document.
25495 new_edits=false is used by the replicator to insert documents
25496 into the target database even if that leads to the creation of
25497 conflicts. Optional
25498 Response Headers
25500 · Content-Type – .INDENT 2.0
25502 · application/json
25504 · text/plain; charset=utf-8
25506 · multipart/related
25508 · ETag – Quoted document’s new revision
25511 · Location – Document URI
25513 Response JSON Object
25515 · id (string) – Document ID
25517 · ok (boolean) – Operation status
25519 · rev (string) – Revision MVCC token
25521 Status Codes
25523 · 201 Created – Document created and stored on disk
25525 · 202 Accepted – Document data accepted, but not yet stored on
25527 disk
25528 · 400 Bad Request – Invalid request body or parameters
25530 · 401 Unauthorized – Write privileges required
25532 · 404 Not Found – Specified database or document ID doesn’t
25534 exists
25535 · 409 Conflict – Document with the specified ID already exists
25537 or specified revision is not latest for target document
25538
25540 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
25542 Accept: application/json
25543 Content-Length: 196
25544 Content-Type: application/json
25545 Host: localhost:5984
25546 {
25548 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25549 "ingredients": [
25550 "spaghetti",
25551 "tomato sauce",
25552 "meatballs"
25553 ],
25554 "name": "Spaghetti with meatballs"
25555 }
25556 Response:
25558 HTTP/1.1 201 Created
25560 Cache-Control: must-revalidate
25561 Content-Length: 85
25562 Content-Type: application/json
25563 Date: Wed, 14 Aug 2013 20:31:39 GMT
25564 ETag: "1-917fa2381192822767f010b95b45325b"
25565 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs
25566 Server: CouchDB (Erlang/OTP)
25567 {
25569 "id": "SpaghettiWithMeatballs",
25570 "ok": true,
25571 "rev": "1-917fa2381192822767f010b95b45325b"
25572 }
25573 DELETE /{db}/{docid}
25575 Marks the specified document as deleted by adding a field
25576 _deleted with the value true. Documents with this field will not
25577 be returned within requests anymore, but stay in the database.
25578 You must supply the current (latest) revision, either by using
25579 the rev parameter or by using the If-Match header to specify the
25580 revision.
25581 NOTE:
25583 CouchDB doesn’t completely delete the specified document.
25584 Instead, it leaves a tombstone with very basic information
25585 about the document. The tombstone is required so that the
25586 delete action can be replicated across databases.
25587 SEE ALSO:
25589 Retrieving Deleted Documents
25590 Parameters
25592 · db – Database name
25594 · docid – Document ID
25596 Request Headers
25598 · Accept – .INDENT 2.0
25600 · application/json
25602 · text/plain
25604 · If-Match – Document’s revision. Alternative to rev query
25607 parameter
25608 Query Parameters
25610 · rev (string) – Actual document’s revision
25612 · batch (string) – Stores document in batch mode Possible val‐
25614 ues: ok. Optional
25615 Response Headers
25617 · Content-Type – .INDENT 2.0
25619 · application/json
25621 · text/plain; charset=utf-8
25623 · ETag – Double quoted document’s new revision
25626 Response JSON Object
25628 · id (string) – Document ID
25630 · ok (boolean) – Operation status
25632 · rev (string) – Revision MVCC token
25634 Status Codes
25636 · 200 OK – Document successfully removed
25638 · 202 Accepted – Request was accepted, but changes are not yet
25640 stored on disk
25641 · 400 Bad Request – Invalid request body or parameters
25643 · 401 Unauthorized – Write privileges required
25645 · 404 Not Found – Specified database or document ID doesn’t
25647 exists
25648 · 409 Conflict – Specified revision is not the latest for target
25650 document
25651
25653 DELETE /recipes/FishStew?rev=1-9c65296036141e575d32ba9c034dd3ee HTTP/1.1
25655 Accept: application/json
25656 Host: localhost:5984
25657 Alternatively, instead of rev query parameter you may use
25659 If-Match header:
25660 DELETE /recipes/FishStew HTTP/1.1
25662 Accept: application/json
25663 If-Match: 1-9c65296036141e575d32ba9c034dd3ee
25664 Host: localhost:5984
25665 Response:
25667 HTTP/1.1 200 OK
25669 Cache-Control: must-revalidate
25670 Content-Length: 71
25671 Content-Type: application/json
25672 Date: Wed, 14 Aug 2013 12:23:13 GMT
25673 ETag: "2-056f5f44046ecafc08a2bc2b9c229e20"
25674 Server: CouchDB (Erlang/OTP)
25675 {
25677 "id": "FishStew",
25678 "ok": true,
25679 "rev": "2-056f5f44046ecafc08a2bc2b9c229e20"
25680 }
25681 COPY /{db}/{docid}
25683 The COPY (which is non-standard HTTP) copies an existing docu‐
25684 ment to a new or existing document. Copying a document is only
25685 possible within the same database.
25686 The source document is specified on the request line, with the
25688 Destination header of the request specifying the target docu‐
25689 ment.
25690 Parameters
25692 · db – Database name
25694 · docid – Document ID
25696 Request Headers
25698 · Accept – .INDENT 2.0
25700 · application/json
25702 · text/plain
25704 · Destination – Destination document. Must contain the target
25707 document ID, and optionally the target document revision, if
25708 copying to an existing document. See Copying to an Existing
25709 Document.
25710 · If-Match – Source document’s revision. Alternative to rev
25712 query parameter
25713 Query Parameters
25715 · rev (string) – Revision to copy from. Optional
25717 · batch (string) – Stores document in batch mode Possible val‐
25719 ues: ok. Optional
25720 Response Headers
25722 · Content-Type – .INDENT 2.0
25724 · application/json
25726 · text/plain; charset=utf-8
25728 · ETag – Double quoted document’s new revision
25731 · Location – Document URI
25733 Response JSON Object
25735 · id (string) – Document document ID
25737 · ok (boolean) – Operation status
25739 · rev (string) – Revision MVCC token
25741 Status Codes
25743 · 201 Created – Document successfully created
25745 · 202 Accepted – Request was accepted, but changes are not yet
25747 stored on disk
25748 · 400 Bad Request – Invalid request body or parameters
25750 · 401 Unauthorized – Read or write privileges required
25752 · 404 Not Found – Specified database, document ID or revision
25754 doesn’t exists
25755 · 409 Conflict – Document with the specified ID already exists
25757 or specified revision is not latest for target document
25758
25760 COPY /recipes/SpaghettiWithMeatballs HTTP/1.1
25762 Accept: application/json
25763 Destination: SpaghettiWithMeatballs_Italian
25764 Host: localhost:5984
25765 Response:
25767 HTTP/1.1 201 Created
25769 Cache-Control: must-revalidate
25770 Content-Length: 93
25771 Content-Type: application/json
25772 Date: Wed, 14 Aug 2013 14:21:00 GMT
25773 ETag: "1-e86fdf912560c2321a5fcefc6264e6d9"
25774 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Italian
25775 Server: CouchDB (Erlang/OTP)
25776 {
25778 "id": "SpaghettiWithMeatballs_Italian",
25779 "ok": true,
25780 "rev": "1-e86fdf912560c2321a5fcefc6264e6d9"
25781 }
25782 Attachments
25784 If the document includes attachments, then the returned structure will
25785 contain a summary of the attachments associated with the document, but
25786 not the attachment data itself.
25787 The JSON for the returned document will include the _attachments field,
25789 with one or more attachment definitions.
25790 The _attachments object keys are attachments names while values are
25792 information objects with next structure:
25793 · content_type (string): Attachment MIME type
25795 · data (string): Base64-encoded content. Available if attachment con‐
25797 tent is requested by using the following query parameters:
25798 · attachments=true when querying a document
25800 · attachments=true&include_docs=true when querying a changes feed
25802 or a view
25803 · atts_since.
25805 · digest (string): Content hash digest. It starts with prefix which
25807 announce hash type (md5-) and continues with Base64-encoded hash
25808 digest
25809 · encoded_length (number): Compressed attachment size in bytes. Avail‐
25811 able if content_type is in list of compressible types when the
25812 attachment was added and the following query parameters are speci‐
25813 fied:
25814 · att_encoding_info=true when querying a document
25816 · att_encoding_info=true&include_docs=true when querying a changes
25818 feed or a view
25819 · encoding (string): Compression codec. Available if content_type is in
25821 list of compressible types when the attachment was added and the fol‐
25822 lowing query parameters are specified:
25823 · att_encoding_info=true when querying a document
25825 · att_encoding_info=true&include_docs=true when querying a changes
25827 feed or a view
25828 · length (number): Real attachment size in bytes. Not available if
25830 attachment content requested
25831 · revpos (number): Revision number when attachment was added
25833 · stub (boolean): Has true value if object contains stub info and no
25835 content. Otherwise omitted in response
25836 Basic Attachments Info
25838 Request:
25839 GET /recipes/SpaghettiWithMeatballs HTTP/1.1
25841 Accept: application/json
25842 Host: localhost:5984
25843 Response:
25845 HTTP/1.1 200 OK
25847 Cache-Control: must-revalidate
25848 Content-Length: 660
25849 Content-Type: application/json
25850 Date: Tue, 13 Aug 2013 21:35:37 GMT
25851 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
25852 Server: CouchDB (Erlang/OTP)
25853 {
25855 "_attachments": {
25856 "grandma_recipe.txt": {
25857 "content_type": "text/plain",
25858 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
25859 "length": 1872,
25860 "revpos": 4,
25861 "stub": true
25862 },
25863 "my_recipe.txt": {
25864 "content_type": "text/plain",
25865 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
25866 "length": 85,
25867 "revpos": 5,
25868 "stub": true
25869 },
25870 "photo.jpg": {
25871 "content_type": "image/jpeg",
25872 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
25873 "length": 165504,
25874 "revpos": 2,
25875 "stub": true
25876 }
25877 },
25878 "_id": "SpaghettiWithMeatballs",
25879 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
25880 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25881 "ingredients": [
25882 "spaghetti",
25883 "tomato sauce",
25884 "meatballs"
25885 ],
25886 "name": "Spaghetti with meatballs"
25887 }
25888 Retrieving Attachments Content
25890 It’s possible to retrieve document with all attached files content by
25891 using attachments=true query parameter:
25892 Request:
25894 GET /db/pixel?attachments=true HTTP/1.1
25896 Accept: application/json
25897 Host: localhost:5984
25898 Response:
25900 HTTP/1.1 200 OK
25902 Cache-Control: must-revalidate
25903 Content-Length: 553
25904 Content-Type: application/json
25905 Date: Wed, 14 Aug 2013 11:32:40 GMT
25906 ETag: "4-f1bcae4bf7bbb92310079e632abfe3f4"
25907 Server: CouchDB (Erlang/OTP)
25908 {
25910 "_attachments": {
25911 "pixel.gif": {
25912 "content_type": "image/gif",
25913 "data": "R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7",
25914 "digest": "md5-2JdGiI2i2VELZKnwMers1Q==",
25915 "revpos": 2
25916 },
25917 "pixel.png": {
25918 "content_type": "image/png",
25919 "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKAAAAAXNSR0IArs4c6QAAAANQTFRFAAAAp3o92gAAAAF0Uk5TAEDm2GYAAAABYktHRACIBR1IAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3QgOCx8VHgmcNwAAAApJREFUCNdjYAAAAAIAAeIhvDMAAAAASUVORK5CYII=",
25920 "digest": "md5-Dgf5zxgGuchWrve73evvGQ==",
25921 "revpos": 3
25922 }
25923 },
25924 "_id": "pixel",
25925 "_rev": "4-f1bcae4bf7bbb92310079e632abfe3f4"
25926 }
25927 Or retrieve attached files content since specific revision using
25929 atts_since query parameter:
25930 Request:
25932 GET /recipes/SpaghettiWithMeatballs?atts_since=[%224-874985bc28906155ba0e2e0538f67b05%22] HTTP/1.1
25934 Accept: application/json
25935 Host: localhost:5984
25936 Response:
25938 HTTP/1.1 200 OK
25940 Cache-Control: must-revalidate
25941 Content-Length: 760
25942 Content-Type: application/json
25943 Date: Tue, 13 Aug 2013 21:35:37 GMT
25944 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
25945 Server: CouchDB (Erlang/OTP)
25946 {
25948 "_attachments": {
25949 "grandma_recipe.txt": {
25950 "content_type": "text/plain",
25951 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
25952 "length": 1872,
25953 "revpos": 4,
25954 "stub": true
25955 },
25956 "my_recipe.txt": {
25957 "content_type": "text/plain",
25958 "data": "MS4gQ29vayBzcGFnaGV0dGkKMi4gQ29vayBtZWV0YmFsbHMKMy4gTWl4IHRoZW0KNC4gQWRkIHRvbWF0byBzYXVjZQo1LiAuLi4KNi4gUFJPRklUIQ==",
25959 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
25960 "revpos": 5
25961 },
25962 "photo.jpg": {
25963 "content_type": "image/jpeg",
25964 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
25965 "length": 165504,
25966 "revpos": 2,
25967 "stub": true
25968 }
25969 },
25970 "_id": "SpaghettiWithMeatballs",
25971 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
25972 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
25973 "ingredients": [
25974 "spaghetti",
25975 "tomato sauce",
25976 "meatballs"
25977 ],
25978 "name": "Spaghetti with meatballs"
25979 }
25980 Efficient Multiple Attachments Retrieving
25982 As noted above, retrieving document with attachments=true returns a
25983 large JSON object with all attachments included. When your document
25984 and files are smaller it’s ok, but if you have attached something big‐
25985 ger like media files (audio/video), parsing such response might be very
25986 expensive.
25987 To solve this problem, CouchDB allows to get documents in multi‐
25989 part/related format:
25990 Request:
25992 GET /recipes/secret?attachments=true HTTP/1.1
25994 Accept: multipart/related
25995 Host: localhost:5984
25996 Response:
25998 HTTP/1.1 200 OK
26000 Content-Length: 538
26001 Content-Type: multipart/related; boundary="e89b3e29388aef23453450d10e5aaed0"
26002 Date: Sat, 28 Sep 2013 08:08:22 GMT
26003 ETag: "2-c1c6c44c4bc3c9344b037c8690468605"
26004 Server: CouchDB (Erlang OTP)
26005 --e89b3e29388aef23453450d10e5aaed0
26007 Content-Type: application/json
26008 {"_id":"secret","_rev":"2-c1c6c44c4bc3c9344b037c8690468605","_attachments":{"recipe.txt":{"content_type":"text/plain","revpos":2,"digest":"md5-HV9aXJdEnu0xnMQYTKgOFA==","length":86,"follows":true}}}
26010 --e89b3e29388aef23453450d10e5aaed0
26011 Content-Disposition: attachment; filename="recipe.txt"
26012 Content-Type: text/plain
26013 Content-Length: 86
26014 1. Take R
26016 2. Take E
26017 3. Mix with L
26018 4. Add some A
26019 5. Serve with X
26020 --e89b3e29388aef23453450d10e5aaed0--
26022 In this response the document contains only attachments stub informa‐
26024 tion and quite short while all attachments goes as separate entities
26025 which reduces memory footprint and processing overhead (you’d noticed,
26026 that attachment content goes as raw data, not in base64 encoding,
26027 right?).
26028 Retrieving Attachments Encoding Info
26030 By using att_encoding_info=true query parameter you may retrieve infor‐
26031 mation about compressed attachments size and used codec.
26032 Request:
26034 GET /recipes/SpaghettiWithMeatballs?att_encoding_info=true HTTP/1.1
26036 Accept: application/json
26037 Host: localhost:5984
26038 Response:
26040 HTTP/1.1 200 OK
26042 Cache-Control: must-revalidate
26043 Content-Length: 736
26044 Content-Type: application/json
26045 Date: Tue, 13 Aug 2013 21:35:37 GMT
26046 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26047 Server: CouchDB (Erlang/OTP)
26048 {
26050 "_attachments": {
26051 "grandma_recipe.txt": {
26052 "content_type": "text/plain",
26053 "digest": "md5-Ids41vtv725jyrN7iUvMcQ==",
26054 "encoded_length": 693,
26055 "encoding": "gzip",
26056 "length": 1872,
26057 "revpos": 4,
26058 "stub": true
26059 },
26060 "my_recipe.txt": {
26061 "content_type": "text/plain",
26062 "digest": "md5-198BPPNiT5fqlLxoYYbjBA==",
26063 "encoded_length": 100,
26064 "encoding": "gzip",
26065 "length": 85,
26066 "revpos": 5,
26067 "stub": true
26068 },
26069 "photo.jpg": {
26070 "content_type": "image/jpeg",
26071 "digest": "md5-7Pv4HW2822WY1r/3WDbPug==",
26072 "length": 165504,
26073 "revpos": 2,
26074 "stub": true
26075 }
26076 },
26077 "_id": "SpaghettiWithMeatballs",
26078 "_rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26079 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26080 "ingredients": [
26081 "spaghetti",
26082 "tomato sauce",
26083 "meatballs"
26084 ],
26085 "name": "Spaghetti with meatballs"
26086 }
26087 Creating Multiple Attachments
26089 To create a document with multiple attachments with single request you
26090 need just inline base64 encoded attachments data into the document
26091 body:
26092 {
26094 "_id":"multiple_attachments",
26095 "_attachments":
26096 {
26097 "foo.txt":
26098 {
26099 "content_type":"text\/plain",
26100 "data": "VGhpcyBpcyBhIGJhc2U2NCBlbmNvZGVkIHRleHQ="
26101 },
26102 "bar.txt":
26104 {
26105 "content_type":"text\/plain",
26106 "data": "VGhpcyBpcyBhIGJhc2U2NCBlbmNvZGVkIHRleHQ="
26107 }
26108 }
26109 }
26110 Alternatively, you can upload a document with attachments more effi‐
26112 ciently in multipart/related format. This avoids having to
26113 Base64-encode the attachments, saving CPU and bandwidth. To do this,
26114 set the Content-Type header of the PUT /{db}/{docid} request to multi‐
26115 part/related.
26116 The first MIME body is the document itself, which should have its own
26118 Content-Type of application/json". It also should include an _attach‐
26119 ments metadata object in which each attachment object has a key follows
26120 with value true.
26121 The subsequent MIME bodies are the attachments.
26123 Request:
26125 PUT /temp/somedoc HTTP/1.1
26127 Accept: application/json
26128 Content-Length: 372
26129 Content-Type: multipart/related;boundary="abc123"
26130 Host: localhost:5984
26131 User-Agent: HTTPie/0.6.0
26132 --abc123
26134 Content-Type: application/json
26135 {
26137 "body": "This is a body.",
26138 "_attachments": {
26139 "foo.txt": {
26140 "follows": true,
26141 "content_type": "text/plain",
26142 "length": 21
26143 },
26144 "bar.txt": {
26145 "follows": true,
26146 "content_type": "text/plain",
26147 "length": 20
26148 }
26149 }
26150 }
26151 --abc123
26153 this is 21 chars long
26155 --abc123
26156 this is 20 chars lon
26158 --abc123--
26159 Response:
26161 HTTP/1.1 201 Created
26163 Cache-Control: must-revalidate
26164 Content-Length: 72
26165 Content-Type: application/json
26166 Date: Sat, 28 Sep 2013 09:13:24 GMT
26167 ETag: "1-5575e26acdeb1df561bb5b70b26ba151"
26168 Location: http://localhost:5984/temp/somedoc
26169 Server: CouchDB (Erlang OTP)
26170 {
26172 "id": "somedoc",
26173 "ok": true,
26174 "rev": "1-5575e26acdeb1df561bb5b70b26ba151"
26175 }
26176 Getting a List of Revisions
26178 You can obtain a list of the revisions for a given document by adding
26179 the revs=true parameter to the request URL:
26180 Request:
26182 GET /recipes/SpaghettiWithMeatballs?revs=true HTTP/1.1
26184 Accept: application/json
26185 Host: localhost:5984
26186 Response:
26188 HTTP/1.1 200 OK
26190 Cache-Control: must-revalidate
26191 Content-Length: 584
26192 Content-Type: application/json
26193 Date: Wed, 14 Aug 2013 11:38:26 GMT
26194 ETag: "5-fd96acb3256302bf0dd2f32713161f2a"
26195 Server: CouchDB (Erlang/OTP)
26196 {
26198 "_id": "SpaghettiWithMeatballs",
26199 "_rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26200 "_revisions": {
26201 "ids": [
26202 "6f5ad8db0f34af24a6e0984cd1a6cfb9",
26203 "77fba3a059497f51ec99b9b478b569d2",
26204 "136813b440a00a24834f5cb1ddf5b1f1",
26205 "fd96acb3256302bf0dd2f32713161f2a",
26206 "874985bc28906155ba0e2e0538f67b05",
26207 "0de77a37463bf391d14283e626831f2e",
26208 "d795d1b924777732fdea76538c558b62",
26209 "917fa2381192822767f010b95b45325b"
26210 ],
26211 "start": 8
26212 },
26213 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26214 "ingredients": [
26215 "spaghetti",
26216 "tomato sauce",
26217 "meatballs"
26218 ],
26219 "name": "Spaghetti with meatballs"
26220 }
26221 The returned JSON structure includes the original document, including a
26223 _revisions structure that includes the revision information in next
26224 form:
26225 · ids (array): Array of valid revision IDs, in reverse order (latest
26227 first)
26228 · start (number): Prefix number for the latest revision
26230 Obtaining an Extended Revision History
26232 You can get additional information about the revisions for a given doc‐
26233 ument by supplying the revs_info argument to the query:
26234 Request:
26236 GET /recipes/SpaghettiWithMeatballs?revs_info=true HTTP/1.1
26238 Accept: application/json
26239 Host: localhost:5984
26240 Response:
26242 HTTP/1.1 200 OK
26244 Cache-Control: must-revalidate
26245 Content-Length: 802
26246 Content-Type: application/json
26247 Date: Wed, 14 Aug 2013 11:40:55 GMT
26248 Server: CouchDB (Erlang/OTP)
26249 {
26251 "_id": "SpaghettiWithMeatballs",
26252 "_rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26253 "_revs_info": [
26254 {
26255 "rev": "8-6f5ad8db0f34af24a6e0984cd1a6cfb9",
26256 "status": "available"
26257 },
26258 {
26259 "rev": "7-77fba3a059497f51ec99b9b478b569d2",
26260 "status": "deleted"
26261 },
26262 {
26263 "rev": "6-136813b440a00a24834f5cb1ddf5b1f1",
26264 "status": "available"
26265 },
26266 {
26267 "rev": "5-fd96acb3256302bf0dd2f32713161f2a",
26268 "status": "missing"
26269 },
26270 {
26271 "rev": "4-874985bc28906155ba0e2e0538f67b05",
26272 "status": "missing"
26273 },
26274 {
26275 "rev": "3-0de77a37463bf391d14283e626831f2e",
26276 "status": "missing"
26277 },
26278 {
26279 "rev": "2-d795d1b924777732fdea76538c558b62",
26280 "status": "missing"
26281 },
26282 {
26283 "rev": "1-917fa2381192822767f010b95b45325b",
26284 "status": "missing"
26285 }
26286 ],
26287 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26288 "ingredients": [
26289 "spaghetti",
26290 "tomato sauce",
26291 "meatballs"
26292 ],
26293 "name": "Spaghetti with meatballs"
26294 }
26295 The returned document contains _revs_info field with extended revision
26297 information, including the availability and status of each revision.
26298 This array field contains objects with following structure:
26299 · rev (string): Full revision string
26301 · status (string): Status of the revision. Maybe one of:
26303 · available: Revision is available for retrieving with rev query
26305 parameter
26306 · missing: Revision is not available
26308 · deleted: Revision belongs to deleted document
26310 Obtaining a Specific Revision
26312 To get a specific revision, use the rev argument to the request, and
26313 specify the full revision number. The specified revision of the docu‐
26314 ment will be returned, including a _rev field specifying the revision
26315 that was requested.
26316 Request:
26318 GET /recipes/SpaghettiWithMeatballs?rev=6-136813b440a00a24834f5cb1ddf5b1f1 HTTP/1.1
26320 Accept: application/json
26321 Host: localhost:5984
26322 Response:
26324 HTTP/1.1 200 OK
26326 Cache-Control: must-revalidate
26327 Content-Length: 271
26328 Content-Type: application/json
26329 Date: Wed, 14 Aug 2013 11:40:55 GMT
26330 Server: CouchDB (Erlang/OTP)
26331 {
26333 "_id": "SpaghettiWithMeatballs",
26334 "_rev": "6-136813b440a00a24834f5cb1ddf5b1f1",
26335 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26336 "ingredients": [
26337 "spaghetti",
26338 "tomato sauce",
26339 "meatballs"
26340 ],
26341 "name": "Spaghetti with meatballs"
26342 }
26343 Retrieving Deleted Documents
26345 CouchDB doesn’t actually delete documents via DELETE /{db}/{docid}.
26346 Instead, it leaves tombstone with very basic information about the doc‐
26347 ument. If you just GET /{db}/{docid} CouchDB returns 404 Not Found
26348 response:
26349 Request:
26351 GET /recipes/FishStew HTTP/1.1
26353 Accept: application/json
26354 Host: localhost:5984
26355 Response:
26357 HTTP/1.1 404 Object Not Found
26359 Cache-Control: must-revalidate
26360 Content-Length: 41
26361 Content-Type: application/json
26362 Date: Wed, 14 Aug 2013 12:23:27 GMT
26363 Server: CouchDB (Erlang/OTP)
26364 {
26366 "error": "not_found",
26367 "reason": "deleted"
26368 }
26369 However, you may retrieve document’s tombstone by using rev query
26371 parameter with GET /{db}/{docid} request:
26372 Request:
26374 GET /recipes/FishStew?rev=2-056f5f44046ecafc08a2bc2b9c229e20 HTTP/1.1
26376 Accept: application/json
26377 Host: localhost:5984
26378 Response:
26380 HTTP/1.1 200 OK
26382 Cache-Control: must-revalidate
26383 Content-Length: 79
26384 Content-Type: application/json
26385 Date: Wed, 14 Aug 2013 12:30:22 GMT
26386 ETag: "2-056f5f44046ecafc08a2bc2b9c229e20"
26387 Server: CouchDB (Erlang/OTP)
26388 {
26390 "_deleted": true,
26391 "_id": "FishStew",
26392 "_rev": "2-056f5f44046ecafc08a2bc2b9c229e20"
26393 }
26394 Updating an Existing Document
26396 To update an existing document you must specify the current revision
26397 number within the _rev parameter.
26398 Request:
26400 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
26402 Accept: application/json
26403 Content-Length: 258
26404 Content-Type: application/json
26405 Host: localhost:5984
26406 {
26408 "_rev": "1-917fa2381192822767f010b95b45325b",
26409 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26410 "ingredients": [
26411 "spaghetti",
26412 "tomato sauce",
26413 "meatballs"
26414 ],
26415 "name": "Spaghetti with meatballs",
26416 "serving": "hot"
26417 }
26418 Alternatively, you can supply the current revision number in the
26420 If-Match HTTP header of the request:
26421 PUT /recipes/SpaghettiWithMeatballs HTTP/1.1
26423 Accept: application/json
26424 Content-Length: 258
26425 Content-Type: application/json
26426 If-Match: 1-917fa2381192822767f010b95b45325b
26427 Host: localhost:5984
26428 {
26430 "description": "An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.",
26431 "ingredients": [
26432 "spaghetti",
26433 "tomato sauce",
26434 "meatballs"
26435 ],
26436 "name": "Spaghetti with meatballs",
26437 "serving": "hot"
26438 }
26439 Response:
26441 HTTP/1.1 201 Created
26443 Cache-Control: must-revalidate
26444 Content-Length: 85
26445 Content-Type: application/json
26446 Date: Wed, 14 Aug 2013 20:33:56 GMT
26447 ETag: "2-790895a73b63fb91dd863388398483dd"
26448 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs
26449 Server: CouchDB (Erlang/OTP)
26450 {
26452 "id": "SpaghettiWithMeatballs",
26453 "ok": true,
26454 "rev": "2-790895a73b63fb91dd863388398483dd"
26455 }
26456 Copying from a Specific Revision
26458 To copy from a specific version, use the rev argument to the query
26459 string or If-Match:
26460 Request:
26462 COPY /recipes/SpaghettiWithMeatballs HTTP/1.1
26464 Accept: application/json
26465 Destination: SpaghettiWithMeatballs_Original
26466 If-Match: 1-917fa2381192822767f010b95b45325b
26467 Host: localhost:5984
26468 Response:
26470 HTTP/1.1 201 Created
26472 Cache-Control: must-revalidate
26473 Content-Length: 93
26474 Content-Type: application/json
26475 Date: Wed, 14 Aug 2013 14:21:00 GMT
26476 ETag: "1-917fa2381192822767f010b95b45325b"
26477 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Original
26478 Server: CouchDB (Erlang/OTP)
26479 {
26481 "id": "SpaghettiWithMeatballs_Original",
26482 "ok": true,
26483 "rev": "1-917fa2381192822767f010b95b45325b"
26484 }
26485 Copying to an Existing Document
26487 To copy to an existing document, you must specify the current revision
26488 string for the target document by appending the rev parameter to the
26489 Destination header string.
26490 Request:
26492 COPY /recipes/SpaghettiWithMeatballs?rev=8-6f5ad8db0f34af24a6e0984cd1a6cfb9 HTTP/1.1
26494 Accept: application/json
26495 Destination: SpaghettiWithMeatballs_Original?rev=1-917fa2381192822767f010b95b45325b
26496 Host: localhost:5984
26497 Response:
26499 HTTP/1.1 201 Created
26501 Cache-Control: must-revalidate
26502 Content-Length: 93
26503 Content-Type: application/json
26504 Date: Wed, 14 Aug 2013 14:21:00 GMT
26505 ETag: "2-62e778c9ec09214dd685a981dcc24074""
26506 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs_Original
26507 Server: CouchDB (Erlang/OTP)
26508 {
26510 "id": "SpaghettiWithMeatballs_Original",
26511 "ok": true,
26512 "rev": "2-62e778c9ec09214dd685a981dcc24074"
26513 }
26514 /db/doc/attachment
26516 HEAD /{db}/{docid}/{attname}
26517 Returns the HTTP headers containing a minimal amount of informa‐
26518 tion about the specified attachment. The method supports the
26519 same query arguments as the GET /{db}/{docid}/{attname} method,
26520 but only the header information (including attachment size,
26521 encoding and the MD5 hash as an ETag), is returned.
26522 Parameters
26524 · db – Database name
26526 · docid – Document ID
26528 · attname – Attachment name
26530 Request Headers
26532 · If-Match – Document’s revision. Alternative to rev
26534 query parameter
26535 · If-None-Match – Attachment’s base64 encoded MD5 binary
26537 digest. Optional
26538 Query Parameters
26540 · rev (string) – Document’s revision. Optional
26542 Response Headers
26544 · Accept-Ranges – Range request aware. Used for attach‐
26546 ments with application/octet-stream content type
26547 · Content-Encoding – Used compression codec. Available if
26549 attachment’s content_type is in list of compressible
26550 types
26551 · Content-Length – Attachment size. If compression codec
26553 was used, this value is about compressed size, not
26554 actual
26555 · ETag – Double quoted base64 encoded MD5 binary digest
26557 Status Codes
26559 · 200 OK – Attachment exists
26561 · 401 Unauthorized – Read privilege required
26563 · 404 Not Found – Specified database, document or attach‐
26565 ment was not found
26566 Request:
26568 HEAD /recipes/SpaghettiWithMeatballs/recipe.txt HTTP/1.1
26570 Host: localhost:5984
26571 Response:
26573 HTTP/1.1 200 OK
26575 Accept-Ranges: none
26576 Cache-Control: must-revalidate
26577 Content-Encoding: gzip
26578 Content-Length: 100
26579 Content-Type: text/plain
26580 Date: Thu, 15 Aug 2013 12:42:42 GMT
26581 ETag: "vVa/YgiE1+Gh0WfoFJAcSg=="
26582 Server: CouchDB (Erlang/OTP)
26583 GET /{db}/{docid}/{attname}
26585 Returns the file attachment associated with the document. The
26586 raw data of the associated attachment is returned (just as if
26587 you were accessing a static file. The returned Content-Type will
26588 be the same as the content type set when the document attachment
26589 was submitted into the database.
26590 Parameters
26592 · db – Database name
26594 · docid – Document ID
26596 · attname – Attachment name
26598 Request Headers
26600 · If-Match – Document’s revision. Alternative to rev
26602 query parameter
26603 · If-None-Match – Attachment’s base64 encoded MD5 binary
26605 digest. Optional
26606 Query Parameters
26608 · rev (string) – Document’s revision. Optional
26610 Response Headers
26612 · Accept-Ranges – Range request aware. Used for attach‐
26614 ments with application/octet-stream
26615 · Content-Encoding – Used compression codec. Available if
26617 attachment’s content_type is in list of compressible
26618 types
26619 · Content-Length – Attachment size. If compression codec
26621 is used, this value is about compressed size, not
26622 actual
26623 · ETag – Double quoted base64 encoded MD5 binary digest
26625 Response
26627 Stored content
26628 Status Codes
26630 · 200 OK – Attachment exists
26632 · 401 Unauthorized – Read privilege required
26634 · 404 Not Found – Specified database, document or attach‐
26636 ment was not found
26637 PUT /{db}/{docid}/{attname}
26639 Uploads the supplied content as an attachment to the specified
26640 document. The attachment name provided must be a URL encoded
26641 string. You must supply the Content-Type header, and for an
26642 existing document you must also supply either the rev query
26643 argument or the If-Match HTTP header. If the revision is omit‐
26644 ted, a new, otherwise empty document will be created with the
26645 provided attachment, or a conflict will occur.
26646 If case when uploading an attachment using an existing attach‐
26648 ment name, CouchDB will update the corresponding stored content
26649 of the database. Since you must supply the revision information
26650 to add an attachment to the document, this serves as validation
26651 to update the existing attachment.
26652 NOTE:
26654 Uploading an attachment updates the corresponding document
26655 revision. Revisions are tracked for the parent document, not
26656 individual attachments.
26657 Parameters
26659 · db – Database name
26661 · docid – Document ID
26663 · attname – Attachment name
26665 Request Headers
26667 · Content-Type – Attachment MIME type. Required
26669 · If-Match – Document revision. Alternative to rev query
26671 parameter
26672 Query Parameters
26674 · rev (string) – Document revision. Optional
26676 Response JSON Object
26678 · id (string) – Document ID
26680 · ok (boolean) – Operation status
26682 · rev (string) – Revision MVCC token
26684 Status Codes
26686 · 201 Created – Attachment created and stored on disk
26688 · 202 Accepted – Request was accepted, but changes are
26690 not yet stored on disk
26691 · 400 Bad Request – Invalid request body or parameters
26693 · 401 Unauthorized – Write privileges required
26695 · 404 Not Found – Specified database, document or attach‐
26697 ment was not found
26698 · 409 Conflict – Document’s revision wasn’t specified or
26700 it’s not the latest
26701 Request:
26703 PUT /recipes/SpaghettiWithMeatballs/recipe.txt HTTP/1.1
26705 Accept: application/json
26706 Content-Length: 86
26707 Content-Type: text/plain
26708 Host: localhost:5984
26709 If-Match: 1-917fa2381192822767f010b95b45325b
26710 1. Cook spaghetti
26712 2. Cook meatballs
26713 3. Mix them
26714 4. Add tomato sauce
26715 5. ...
26716 6. PROFIT!
26717 Response:
26719 HTTP/1.1 201 Created
26721 Cache-Control: must-revalidate
26722 Content-Length: 85
26723 Content-Type: application/json
26724 Date: Thu, 15 Aug 2013 12:38:04 GMT
26725 ETag: "2-ce91aed0129be8f9b0f650a2edcfd0a4"
26726 Location: http://localhost:5984/recipes/SpaghettiWithMeatballs/recipe.txt
26727 Server: CouchDB (Erlang/OTP)
26728 {
26730 "id": "SpaghettiWithMeatballs",
26731 "ok": true,
26732 "rev": "2-ce91aed0129be8f9b0f650a2edcfd0a4"
26733 }
26734 DELETE /{db}/{docid}/{attname}
26736 Deletes the attachment with filename {attname} of the specified
26737 doc. You must supply the rev query parameter or If-Match with
26738 the current revision to delete the attachment.
26739 NOTE:
26741 Deleting an attachment updates the corresponding document
26742 revision. Revisions are tracked for the parent document, not
26743 individual attachments.
26744 Parameters
26746 · db – Database name
26748 · docid – Document ID
26750 Request Headers
26752 · Accept – .INDENT 2.0
26754 · application/json
26756 · text/plain
26758 · If-Match – Document revision. Alternative to rev query parame‐
26761 ter
26762 Query Parameters
26764 · rev (string) – Document revision. Required
26766 · batch (string) – Store changes in batch mode Possible values:
26768 ok. Optional
26769 Response Headers
26771 · Content-Type – .INDENT 2.0
26773 · application/json
26775 · text/plain; charset=utf-8
26777 · ETag – Double quoted document’s new revision
26780 Response JSON Object
26782 · id (string) – Document ID
26784 · ok (boolean) – Operation status
26786 · rev (string) – Revision MVCC token
26788 Status Codes
26790 · 200 OK – Attachment successfully removed
26792 · 202 Accepted – Request was accepted, but changes are not yet
26794 stored on disk
26795 · 400 Bad Request – Invalid request body or parameters
26797 · 401 Unauthorized – Write privileges required
26799 · 404 Not Found – Specified database, document or attachment was
26801 not found
26802 · 409 Conflict – Document’s revision wasn’t specified or it’s
26804 not the latest
26805
26807 DELETE /recipes/SpaghettiWithMeatballs?rev=6-440b2dd39c20413045748b42c6aba6e2 HTTP/1.1
26809 Accept: application/json
26810 Host: localhost:5984
26811 Alternatively, instead of rev query parameter you may use
26813 If-Match header:
26814 DELETE /recipes/SpaghettiWithMeatballs HTTP/1.1
26816 Accept: application/json
26817 If-Match: 6-440b2dd39c20413045748b42c6aba6e2
26818 Host: localhost:5984
26819 Response:
26821 HTTP/1.1 200 OK
26823 Cache-Control: must-revalidate
26824 Content-Length: 85
26825 Content-Type: application/json
26826 Date: Wed, 14 Aug 2013 12:23:13 GMT
26827 ETag: "7-05185cf5fcdf4b6da360af939431d466"
26828 Server: CouchDB (Erlang/OTP)
26829 {
26831 "id": "SpaghettiWithMeatballs",
26832 "ok": true,
26833 "rev": "7-05185cf5fcdf4b6da360af939431d466"
26834 }
26835 HTTP Range Requests
26837 HTTP allows you to specify byte ranges for requests. This allows the
26838 implementation of resumable downloads and skippable audio and video
26839 streams alike. This is available for all attachments inside CouchDB.
26840 This is just a real quick run through how this looks under the hood.
26842 Usually, you will have larger binary files to serve from CouchDB, like
26843 MP3s and videos, but to make things a little more obvious, I use a text
26844 file here (Note that I use the application/octet-stream :header`Con‐
26845 tent-Type` instead of text/plain).
26846 shell> cat file.txt
26848 My hovercraft is full of eels!
26849 Now let’s store this text file as an attachment in CouchDB. First, we
26851 create a database:
26852 shell> curl -X PUT http://127.0.0.1:5984/test
26854 {"ok":true}
26855 Then we create a new document and the file attachment in one go:
26857 shell> curl -X PUT http://127.0.0.1:5984/test/doc/file.txt \
26859 -H "Content-Type: application/octet-stream" -d@file.txt
26860 {"ok":true,"id":"doc","rev":"1-287a28fa680ae0c7fb4729bf0c6e0cf2"}
26861 Now we can request the whole file easily:
26863 shell> curl -X GET http://127.0.0.1:5984/test/doc/file.txt
26865 My hovercraft is full of eels!
26866 But say we only want the first 13 bytes:
26868 shell> curl -X GET http://127.0.0.1:5984/test/doc/file.txt \
26870 -H "Range: bytes=0-12"
26871 My hovercraft
26872 HTTP supports many ways to specify single and even multiple byte
26874 ranges. Read all about it in RFC 2616#section-14.27.
26875 NOTE:
26877 Databases that have been created with CouchDB 1.0.2 or earlier will
26878 support range requests in 3.0, but they are using a less-optimal
26879 algorithm. If you plan to make heavy use of this feature, make sure
26880 to compact your database with CouchDB 3.0 to take advantage of a
26881 better algorithm to find byte ranges.
26882 Design Documents
26884 In CouchDB, design documents provide the main interface for building a
26885 CouchDB application. The design document defines the views used to
26886 extract information from CouchDB through one or more views. Design doc‐
26887 uments are created within your CouchDB instance in the same way as you
26888 create database documents, but the content and definition of the docu‐
26889 ments is different. Design Documents are named using an ID defined with
26890 the design document URL path, and this URL can then be used to access
26891 the database contents.
26892 Views and lists operate together to provide automated (and formatted)
26894 output from your database.
26895 /db/_design/design-doc
26897 HEAD /{db}/_design/{ddoc}
26898 Returns the HTTP Headers containing a minimal amount of informa‐
26899 tion about the specified design document.
26900 SEE ALSO:
26902 HEAD /{db}/{docid}
26903 GET /{db}/_design/{ddoc}
26905 Returns the contents of the design document specified with the
26906 name of the design document and from the specified database from
26907 the URL. Unless you request a specific revision, the latest
26908 revision of the document will always be returned.
26909 SEE ALSO:
26911 GET /{db}/{docid}
26912 PUT /{db}/_design/{ddoc}
26914 The PUT method creates a new named design document, or creates a
26915 new revision of the existing design document.
26916 The design documents have some agreement upon their fields and
26918 structure. Currently it is the following:
26919 · language (string): Defines Query Server to process design doc‐
26921 ument functions
26922 · options (object): View’s default options
26924 · filters (object): Filter functions definition
26926 · lists (object): List functions definition. Deprecated.
26928 · rewrites (array or string): Rewrite rules definition. Depre‐
26930 cated.
26931 · shows (object): Show functions definition. Deprecated.
26933 · updates (object): Update functions definition
26935 · validate_doc_update (string): Validate document update func‐
26937 tion source
26938 · views (object): View functions definition.
26940 · autoupdate (boolean): Indicates whether to automatically build
26942 indexes defined in this design document. Default is true.
26943 Note, that for filters, lists, shows and updates fields objects
26945 are mapping of function name to string function source code. For
26946 views mapping is the same except that values are objects with
26947 map and reduce (optional) keys which also contains functions
26948 source code.
26949 SEE ALSO:
26951 PUT /{db}/{docid}
26952 DELETE /{db}/_design/{ddoc}
26954 Deletes the specified document from the database. You must sup‐
26955 ply the current (latest) revision, either by using the rev
26956 parameter to specify the revision.
26957 SEE ALSO:
26959 DELETE /{db}/{docid}
26960 COPY /{db}/_design/{ddoc}
26962 The COPY (which is non-standard HTTP) copies an existing design
26963 document to a new or existing one.
26964 Given that view indexes on disk are named after their MD5 hash
26966 of the view definition, and that a COPY operation won’t actually
26967 change that definition, the copied views won’t have to be recon‐
26968 structed. Both views will be served from the same index on
26969 disk.
26970 SEE ALSO:
26972 COPY /{db}/{docid}
26973 /db/_design/design-doc/attachment
26975 HEAD /{db}/_design/{ddoc}/{attname}
26976 Returns the HTTP headers containing a minimal amount of informa‐
26977 tion about the specified attachment.
26978 SEE ALSO:
26980 HEAD /{db}/{docid}/{attname}
26981 GET /{db}/_design/{ddoc}/{attname}
26983 Returns the file attachment associated with the design document.
26984 The raw data of the associated attachment is returned (just as
26985 if you were accessing a static file.
26986 SEE ALSO:
26988 GET /{db}/{docid}/{attname}
26989 PUT /{db}/_design/{ddoc}/{attname}
26991 Uploads the supplied content as an attachment to the specified
26992 design document. The attachment name provided must be a URL
26993 encoded string.
26994 SEE ALSO:
26996 PUT /{db}/{docid}/{attname}
26997 DELETE /{db}/_design/{ddoc}/{attname}
26999 Deletes the attachment of the specified design document.
27000 SEE ALSO:
27002 DELETE /{db}/{docid}/{attname}
27003 /db/_design/design-doc/_info
27005 GET /{db}/_design/{ddoc}/_info
27006 Obtains information about the specified design document, includ‐
27007 ing the index, index size and current status of the design docu‐
27008 ment and associated index information.
27009 Parameters
27011 · db – Database name
27013 · ddoc – Design document name
27015 Request Headers
27017 · Accept – .INDENT 2.0
27019 · application/json
27021 · text/plain
27023 Response Headers
27026 · Content-Type – .INDENT 2.0
27028 · application/json
27030 · text/plain; charset=utf-8
27032 Response JSON Object
27035 · name (string) – Design document name
27037 · view_index (object) – View Index Information
27039 Status Codes
27041 · 200 OK – Request completed successfully
27043
27045 GET /recipes/_design/recipe/_info HTTP/1.1
27047 Accept: application/json
27048 Host: localhost:5984
27049 Response:
27051 HTTP/1.1 200 OK
27053 Cache-Control: must-revalidate
27054 Content-Length: 263
27055 Content-Type: application/json
27056 Date: Sat, 17 Aug 2013 12:54:17 GMT
27057 Server: CouchDB (Erlang/OTP)
27058 {
27060 "name": "recipe",
27061 "view_index": {
27062 "compact_running": false,
27063 "language": "python",
27064 "purge_seq": 0,
27065 "signature": "a59a1bb13fdf8a8a584bc477919c97ac",
27066 "sizes": {
27067 "active": 926691,
27068 "disk": 1982704,
27069 "external": 1535701
27070 },
27071 "update_seq": 12397,
27072 "updater_running": false,
27073 "waiting_clients": 0,
27074 "waiting_commit": false
27075 }
27076 }
27077 View Index Information
27079 The response from GET /{db}/_design/{ddoc}/_info contains view_index
27080 (object) field with the next structure:
27081 · compact_running (boolean): Indicates whether a compaction routine is
27083 currently running on the view
27084 · sizes.active (number): The size of live data inside the view, in
27086 bytes
27087 · sizes.external (number): The uncompressed size of view contents in
27089 bytes
27090 · sizes.file (number): Size in bytes of the view as stored on disk
27092 · language (string): Language for the defined views
27094 · purge_seq (number): The purge sequence that has been processed
27096 · signature (string): MD5 signature of the views for the design docu‐
27098 ment
27099 · update_seq (number / string): The update sequence of the correspond‐
27101 ing database that has been indexed
27102 · updater_running (boolean): Indicates if the view is currently being
27104 updated
27105 · waiting_clients (number): Number of clients waiting on views from
27107 this design document
27108 · waiting_commit (boolean): Indicates if there are outstanding commits
27110 to the underlying database that need to processed
27111 /db/_design/design-doc/_view/view-name
27113 GET /{db}/_design/{ddoc}/_view/{view}
27114 Executes the specified view function from the specified design
27115 document.
27116 Parameters
27118 · db – Database name
27120 · ddoc – Design document name
27122 · view – View function name
27124 Request Headers
27126 · Accept – .INDENT 2.0
27128 · application/json
27130 · text/plain
27132 Query Parameters
27135 · conflicts (boolean) – Include conflicts information in
27137 response. Ignored if include_docs isn’t true. Default is
27138 false.
27139 · descending (boolean) – Return the documents in descending
27141 order by key. Default is false.
27142 · endkey (json) – Stop returning records when the specified key
27144 is reached.
27145 · end_key (json) – Alias for endkey param
27147 · endkey_docid (string) – Stop returning records when the speci‐
27149 fied document ID is reached. Ignored if endkey is not set.
27150 · end_key_doc_id (string) – Alias for endkey_docid.
27152 · group (boolean) – Group the results using the reduce function
27154 to a group or single row. Implies reduce is true and the maxi‐
27155 mum group_level. Default is false.
27156 · group_level (number) – Specify the group level to be used.
27158 Implies group is true.
27159 · include_docs (boolean) – Include the associated document with
27161 each row. Default is false.
27162 · attachments (boolean) – Include the Base64-encoded content of
27164 attachments in the documents that are included if include_docs
27165 is true. Ignored if include_docs isn’t true. Default is false.
27166 · att_encoding_info (boolean) – Include encoding information in
27168 attachment stubs if include_docs is true and the particular
27169 attachment is compressed. Ignored if include_docs isn’t true.
27170 Default is false.
27171 · inclusive_end (boolean) – Specifies whether the specified end
27173 key should be included in the result. Default is true.
27174 · key (json) – Return only documents that match the specified
27176 key.
27177 · keys (json-array) – Return only documents where the key
27179 matches one of the keys specified in the array.
27180 · limit (number) – Limit the number of the returned documents to
27182 the specified number.
27183 · reduce (boolean) – Use the reduction function. Default is true
27185 when a reduce function is defined.
27186 · skip (number) – Skip this number of records before starting to
27188 return the results. Default is 0.
27189 · sorted (boolean) – Sort returned rows (see Sorting Returned
27191 Rows). Setting this to false offers a performance boost. The
27192 total_rows and offset fields are not available when this is
27193 set to false. Default is true.
27194 · stable (boolean) – Whether or not the view results should be
27196 returned from a stable set of shards. Default is false.
27197 · stale (string) – Allow the results from a stale view to be
27199 used. Supported values: ok, update_after and false. ok is
27200 equivalent to stable=true&update=false. update_after is
27201 equivalent to stable=true&update=lazy. false is equivalent to
27202 stable=false&update=true.
27203 · startkey (json) – Return records starting with the specified
27205 key.
27206 · start_key (json) – Alias for startkey.
27208 · startkey_docid (string) – Return records starting with the
27210 specified document ID. Ignored if startkey is not set.
27211 · start_key_doc_id (string) – Alias for startkey_docid param
27213 · update (string) – Whether or not the view in question should
27215 be updated prior to responding to the user. Supported values:
27216 true, false, lazy. Default is true.
27217 · update_seq (boolean) – Whether to include in the response an
27219 update_seq value indicating the sequence id of the database
27220 the view reflects. Default is false.
27221 Response Headers
27223 · Content-Type – .INDENT 2.0
27225 · application/json
27227 · text/plain; charset=utf-8
27229 · ETag – Response signature
27232 · Transfer-Encoding – chunked
27234 Response JSON Object
27236 · offset (number) – Offset where the document list started.
27238 · rows (array) – Array of view row objects. By default the
27240 information returned contains only the document ID and revi‐
27241 sion.
27242 · total_rows (number) – Number of documents in the data‐
27244 base/view.
27245 · update_seq (object) – Current update sequence for the data‐
27247 base.
27248 Status Codes
27250 · 200 OK – Request completed successfully
27252 · 400 Bad Request – Invalid request
27254 · 401 Unauthorized – Read permission required
27256 · 404 Not Found – Specified database, design document or view is
27258 missed
27259
27261 GET /recipes/_design/ingredients/_view/by_name HTTP/1.1
27263 Accept: application/json
27264 Host: localhost:5984
27265 Response:
27267 HTTP/1.1 200 OK
27269 Cache-Control: must-revalidate
27270 Content-Type: application/json
27271 Date: Wed, 21 Aug 2013 09:12:06 GMT
27272 ETag: "2FOLSBSW4O6WB798XU4AQYA9B"
27273 Server: CouchDB (Erlang/OTP)
27274 Transfer-Encoding: chunked
27275 {
27277 "offset": 0,
27278 "rows": [
27279 {
27280 "id": "SpaghettiWithMeatballs",
27281 "key": "meatballs",
27282 "value": 1
27283 },
27284 {
27285 "id": "SpaghettiWithMeatballs",
27286 "key": "spaghetti",
27287 "value": 1
27288 },
27289 {
27290 "id": "SpaghettiWithMeatballs",
27291 "key": "tomato sauce",
27292 "value": 1
27293 }
27294 ],
27295 "total_rows": 3
27296 }
27297Changed in version 1.6.0: added attachments and att_encoding_info parameters
27299Changed in version 2.0.0: added sorted parameter
27302Changed in version 2.1.0: added stable and update parameters
27305
27308 Using the attachments parameter to include attachments in view
27309 results is not recommended for large attachment sizes. Also note
27310 that the Base64-encoding that is used leads to a 33% overhead (i.e.
27311 one third) in transfer size for attachments.
27312 POST /{db}/_design/{ddoc}/_view/{view}
27314 Executes the specified view function from the specified design
27315 document. POST view functionality supports identical parameters
27316 and behavior as specified in the GET
27317 /{db}/_design/{ddoc}/_view/{view} API but allows for the query
27318 string parameters to be supplied as keys in a JSON object in the
27319 body of the POST request.
27320 Request:
27322 POST /recipes/_design/ingredients/_view/by_name HTTP/1.1
27324 Accept: application/json
27325 Content-Length: 37
27326 Host: localhost:5984
27327 {
27329 "keys": [
27330 "meatballs",
27331 "spaghetti"
27332 ]
27333 }
27334 Response:
27336 HTTP/1.1 200 OK
27338 Cache-Control: must-revalidate
27339 Content-Type: application/json
27340 Date: Wed, 21 Aug 2013 09:14:13 GMT
27341 ETag: "6R5NM8E872JIJF796VF7WI3FZ"
27342 Server: CouchDB (Erlang/OTP)
27343 Transfer-Encoding: chunked
27344 {
27346 "offset": 0,
27347 "rows": [
27348 {
27349 "id": "SpaghettiWithMeatballs",
27350 "key": "meatballs",
27351 "value": 1
27352 },
27353 {
27354 "id": "SpaghettiWithMeatballs",
27355 "key": "spaghetti",
27356 "value": 1
27357 }
27358 ],
27359 "total_rows": 3
27360 }
27361 View Options
27363 There are two view indexing options that can be defined in a design
27364 document as boolean properties of an options object. Unlike the others
27365 querying options, these aren’t URL parameters because they take effect
27366 when the view index is generated, not when it’s accessed:
27367 · local_seq (boolean): Makes documents’ local sequence numbers avail‐
27369 able to map functions (as a _local_seq document property)
27370 · include_design (boolean): Allows map functions to be called on design
27372 documents as well as regular documents
27373 Querying Views and Indexes
27375 The definition of a view within a design document also creates an index
27376 based on the key information defined within each view. The production
27377 and use of the index significantly increases the speed of access and
27378 searching or selecting documents from the view.
27379 However, the index is not updated when new documents are added or modi‐
27381 fied in the database. Instead, the index is generated or updated,
27382 either when the view is first accessed, or when the view is accessed
27383 after a document has been updated. In each case, the index is updated
27384 before the view query is executed against the database.
27385 View indexes are updated incrementally in the following situations:
27387 · A new document has been added to the database.
27389 · A document has been deleted from the database.
27391 · A document in the database has been updated.
27393 View indexes are rebuilt entirely when the view definition changes. To
27395 achieve this, a ‘fingerprint’ of the view definition is created when
27396 the design document is updated. If the fingerprint changes, then the
27397 view indexes are entirely rebuilt. This ensures that changes to the
27398 view definitions are reflected in the view indexes.
27399 NOTE:
27401 View index rebuilds occur when one view from the same the view group
27402 (i.e. all the views defined within a single a design document) has
27403 been determined as needing a rebuild. For example, if if you have a
27404 design document with different views, and you update the database,
27405 all three view indexes within the design document will be updated.
27406 Because the view is updated when it has been queried, it can result in
27408 a delay in returned information when the view is accessed, especially
27409 if there are a large number of documents in the database and the view
27410 index does not exist. There are a number of ways to mitigate, but not
27411 completely eliminate, these issues. These include:
27412 · Create the view definition (and associated design documents) on your
27414 database before allowing insertion or updates to the documents. If
27415 this is allowed while the view is being accessed, the index can be
27416 updated incrementally.
27417 · Manually force a view request from the database. You can do this
27419 either before users are allowed to use the view, or you can access
27420 the view manually after documents are added or updated.
27421 · Use the changes feed to monitor for changes to the database and then
27423 access the view to force the corresponding view index to be updated.
27424 None of these can completely eliminate the need for the indexes to be
27426 rebuilt or updated when the view is accessed, but they may lessen the
27427 effects on end-users of the index update affecting the user experience.
27428 Another alternative is to allow users to access a ‘stale’ version of
27430 the view index, rather than forcing the index to be updated and dis‐
27431 playing the updated results. Using a stale view may not return the lat‐
27432 est information, but will return the results of the view query using an
27433 existing version of the index.
27434 For example, to access the existing stale view by_recipe in the recipes
27436 design document:
27437 http://localhost:5984/recipes/_design/recipes/_view/by_recipe?stale=ok
27439 Accessing a stale view:
27441 · Does not trigger a rebuild of the view indexes, even if there have
27443 been changes since the last access.
27444 · Returns the current version of the view index, if a current version
27446 exists.
27447 · Returns an empty result set if the given view index does exist.
27449 As an alternative, you use the update_after value to the stale parame‐
27451 ter. This causes the view to be returned as a stale view, but for the
27452 update process to be triggered after the view information has been
27453 returned to the client.
27454 In addition to using stale views, you can also make use of the
27456 update_seq query argument. Using this query argument generates the view
27457 information including the update sequence of the database from which
27458 the view was generated. The returned value can be compared this to the
27459 current update sequence exposed in the database information (returned
27460 by GET /{db}).
27461 Sorting Returned Rows
27463 Each element within the returned array is sorted using native UTF-8
27464 sorting according to the contents of the key portion of the emitted
27465 content. The basic order of output is as follows:
27466 · null
27468 · false
27470 · true
27472 · Numbers
27474 · Text (case sensitive, lowercase first)
27476 · Arrays (according to the values of each element, in order)
27478 · Objects (according to the values of keys, in key order)
27480 Request:
27482 GET /db/_design/test/_view/sorting HTTP/1.1
27484 Accept: application/json
27485 Host: localhost:5984
27486 Response:
27488 HTTP/1.1 200 OK
27490 Cache-Control: must-revalidate
27491 Content-Type: application/json
27492 Date: Wed, 21 Aug 2013 10:09:25 GMT
27493 ETag: "8LA1LZPQ37B6R9U8BK9BGQH27"
27494 Server: CouchDB (Erlang/OTP)
27495 Transfer-Encoding: chunked
27496 {
27498 "offset": 0,
27499 "rows": [
27500 {
27501 "id": "dummy-doc",
27502 "key": null,
27503 "value": null
27504 },
27505 {
27506 "id": "dummy-doc",
27507 "key": false,
27508 "value": null
27509 },
27510 {
27511 "id": "dummy-doc",
27512 "key": true,
27513 "value": null
27514 },
27515 {
27516 "id": "dummy-doc",
27517 "key": 0,
27518 "value": null
27519 },
27520 {
27521 "id": "dummy-doc",
27522 "key": 1,
27523 "value": null
27524 },
27525 {
27526 "id": "dummy-doc",
27527 "key": 10,
27528 "value": null
27529 },
27530 {
27531 "id": "dummy-doc",
27532 "key": 42,
27533 "value": null
27534 },
27535 {
27536 "id": "dummy-doc",
27537 "key": "10",
27538 "value": null
27539 },
27540 {
27541 "id": "dummy-doc",
27542 "key": "hello",
27543 "value": null
27544 },
27545 {
27546 "id": "dummy-doc",
27547 "key": "Hello",
27548 "value": null
27549 },
27550 {
27551 "id": "dummy-doc",
27552 "key": "\u043f\u0440\u0438\u0432\u0435\u0442",
27553 "value": null
27554 },
27555 {
27556 "id": "dummy-doc",
27557 "key": [],
27558 "value": null
27559 },
27560 {
27561 "id": "dummy-doc",
27562 "key": [
27563 1,
27564 2,
27565 3
27566 ],
27567 "value": null
27568 },
27569 {
27570 "id": "dummy-doc",
27571 "key": [
27572 2,
27573 3
27574 ],
27575 "value": null
27576 },
27577 {
27578 "id": "dummy-doc",
27579 "key": [
27580 3
27581 ],
27582 "value": null
27583 },
27584 {
27585 "id": "dummy-doc",
27586 "key": {},
27587 "value": null
27588 },
27589 {
27590 "id": "dummy-doc",
27591 "key": {
27592 "foo": "bar"
27593 },
27594 "value": null
27595 }
27596 ],
27597 "total_rows": 17
27598 }
27599 You can reverse the order of the returned view information by using the
27601 descending query value set to true:
27602 Request:
27604 GET /db/_design/test/_view/sorting?descending=true HTTP/1.1
27606 Accept: application/json
27607 Host: localhost:5984
27608 Response:
27610 HTTP/1.1 200 OK
27612 Cache-Control: must-revalidate
27613 Content-Type: application/json
27614 Date: Wed, 21 Aug 2013 10:09:25 GMT
27615 ETag: "Z4N468R15JBT98OM0AMNSR8U"
27616 Server: CouchDB (Erlang/OTP)
27617 Transfer-Encoding: chunked
27618 {
27620 "offset": 0,
27621 "rows": [
27622 {
27623 "id": "dummy-doc",
27624 "key": {
27625 "foo": "bar"
27626 },
27627 "value": null
27628 },
27629 {
27630 "id": "dummy-doc",
27631 "key": {},
27632 "value": null
27633 },
27634 {
27635 "id": "dummy-doc",
27636 "key": [
27637 3
27638 ],
27639 "value": null
27640 },
27641 {
27642 "id": "dummy-doc",
27643 "key": [
27644 2,
27645 3
27646 ],
27647 "value": null
27648 },
27649 {
27650 "id": "dummy-doc",
27651 "key": [
27652 1,
27653 2,
27654 3
27655 ],
27656 "value": null
27657 },
27658 {
27659 "id": "dummy-doc",
27660 "key": [],
27661 "value": null
27662 },
27663 {
27664 "id": "dummy-doc",
27665 "key": "\u043f\u0440\u0438\u0432\u0435\u0442",
27666 "value": null
27667 },
27668 {
27669 "id": "dummy-doc",
27670 "key": "Hello",
27671 "value": null
27672 },
27673 {
27674 "id": "dummy-doc",
27675 "key": "hello",
27676 "value": null
27677 },
27678 {
27679 "id": "dummy-doc",
27680 "key": "10",
27681 "value": null
27682 },
27683 {
27684 "id": "dummy-doc",
27685 "key": 42,
27686 "value": null
27687 },
27688 {
27689 "id": "dummy-doc",
27690 "key": 10,
27691 "value": null
27692 },
27693 {
27694 "id": "dummy-doc",
27695 "key": 1,
27696 "value": null
27697 },
27698 {
27699 "id": "dummy-doc",
27700 "key": 0,
27701 "value": null
27702 },
27703 {
27704 "id": "dummy-doc",
27705 "key": true,
27706 "value": null
27707 },
27708 {
27709 "id": "dummy-doc",
27710 "key": false,
27711 "value": null
27712 },
27713 {
27714 "id": "dummy-doc",
27715 "key": null,
27716 "value": null
27717 }
27718 ],
27719 "total_rows": 17
27720 }
27721 Sorting order and startkey/endkey
27723 The sorting direction is applied before the filtering applied using the
27724 startkey and endkey query arguments. For example the following query:
27725 GET http://couchdb:5984/recipes/_design/recipes/_view/by_ingredient?startkey=%22carrots%22&endkey=%22egg%22 HTTP/1.1
27727 Accept: application/json
27728 will operate correctly when listing all the matching entries between
27730 carrots and egg. If the order of output is reversed with the descending
27731 query argument, the view request will return no entries:
27732 GET /recipes/_design/recipes/_view/by_ingredient?descending=true&startkey=%22carrots%22&endkey=%22egg%22 HTTP/1.1
27734 Accept: application/json
27735 Host: localhost:5984
27736 {
27738 "total_rows" : 26453,
27739 "rows" : [],
27740 "offset" : 21882
27741 }
27742 The results will be empty because the entries in the view are reversed
27744 before the key filter is applied, and therefore the endkey of “egg”
27745 will be seen before the startkey of “carrots”, resulting in an empty
27746 list.
27747 Instead, you should reverse the values supplied to the startkey and
27749 endkey parameters to match the descending sorting applied to the keys.
27750 Changing the previous example to:
27751 GET /recipes/_design/recipes/_view/by_ingredient?descending=true&startkey=%22egg%22&endkey=%22carrots%22 HTTP/1.1
27753 Accept: application/json
27754 Host: localhost:5984
27755 Raw collation
27757 By default CouchDB using ICU driver for sorting view results. It’s pos‐
27758 sible use binary collation instead for faster view builds where Unicode
27759 collation is not important.
27760 To use raw collation add "collation": "raw" key-value pair to the
27762 design documents options object at the root level. After that, views
27763 will be regenerated and new order applied.
27764 SEE ALSO:
27766 views/collation
27767 Using Limits and Skipping Rows
27769 By default, views return all results. That’s ok when the number of
27770 results is small, but this may lead to problems when there are billions
27771 results, since the client may have to read them all and consume all
27772 available memory.
27773 But it’s possible to reduce output result rows by specifying limit
27775 query parameter. For example, retrieving the list of recipes using the
27776 by_title view and limited to 5 returns only 5 records, while there are
27777 total 2667 records in view:
27778 Request:
27780 GET /recipes/_design/recipes/_view/by_title?limit=5 HTTP/1.1
27782 Accept: application/json
27783 Host: localhost:5984
27784 Response:
27786 HTTP/1.1 200 OK
27788 Cache-Control: must-revalidate
27789 Content-Type: application/json
27790 Date: Wed, 21 Aug 2013 09:14:13 GMT
27791 ETag: "9Q6Q2GZKPH8D5F8L7PB6DBSS9"
27792 Server: CouchDB (Erlang/OTP)
27793 Transfer-Encoding: chunked
27794 {
27796 "offset" : 0,
27797 "rows" : [
27798 {
27799 "id" : "3-tiersalmonspinachandavocadoterrine",
27800 "key" : "3-tier salmon, spinach and avocado terrine",
27801 "value" : [
27802 null,
27803 "3-tier salmon, spinach and avocado terrine"
27804 ]
27805 },
27806 {
27807 "id" : "Aberffrawcake",
27808 "key" : "Aberffraw cake",
27809 "value" : [
27810 null,
27811 "Aberffraw cake"
27812 ]
27813 },
27814 {
27815 "id" : "Adukiandorangecasserole-microwave",
27816 "key" : "Aduki and orange casserole - microwave",
27817 "value" : [
27818 null,
27819 "Aduki and orange casserole - microwave"
27820 ]
27821 },
27822 {
27823 "id" : "Aioli-garlicmayonnaise",
27824 "key" : "Aioli - garlic mayonnaise",
27825 "value" : [
27826 null,
27827 "Aioli - garlic mayonnaise"
27828 ]
27829 },
27830 {
27831 "id" : "Alabamapeanutchicken",
27832 "key" : "Alabama peanut chicken",
27833 "value" : [
27834 null,
27835 "Alabama peanut chicken"
27836 ]
27837 }
27838 ],
27839 "total_rows" : 2667
27840 }
27841 To omit some records you may use skip query parameter:
27843 Request:
27845 GET /recipes/_design/recipes/_view/by_title?limit=3&skip=2 HTTP/1.1
27847 Accept: application/json
27848 Host: localhost:5984
27849 Response:
27851 HTTP/1.1 200 OK
27853 Cache-Control: must-revalidate
27854 Content-Type: application/json
27855 Date: Wed, 21 Aug 2013 09:14:13 GMT
27856 ETag: "H3G7YZSNIVRRHO5FXPE16NJHN"
27857 Server: CouchDB (Erlang/OTP)
27858 Transfer-Encoding: chunked
27859 {
27861 "offset" : 2,
27862 "rows" : [
27863 {
27864 "id" : "Adukiandorangecasserole-microwave",
27865 "key" : "Aduki and orange casserole - microwave",
27866 "value" : [
27867 null,
27868 "Aduki and orange casserole - microwave"
27869 ]
27870 },
27871 {
27872 "id" : "Aioli-garlicmayonnaise",
27873 "key" : "Aioli - garlic mayonnaise",
27874 "value" : [
27875 null,
27876 "Aioli - garlic mayonnaise"
27877 ]
27878 },
27879 {
27880 "id" : "Alabamapeanutchicken",
27881 "key" : "Alabama peanut chicken",
27882 "value" : [
27883 null,
27884 "Alabama peanut chicken"
27885 ]
27886 }
27887 ],
27888 "total_rows" : 2667
27889 }
27890 WARNING:
27892 Using limit and skip parameters is not recommended for results pagi‐
27893 nation. Read pagination recipe why it’s so and how to make it bet‐
27894 ter.
27895 Sending multiple queries to a view
27897 New in version 2.2.
27898 POST /{db}/_design/{ddoc}/_view/{view}/queries
27901 Executes multiple specified view queries against the view func‐
27902 tion from the specified design document.
27903 Parameters
27905 · db – Database name
27907 · ddoc – Design document name
27909 · view – View function name
27911 Request Headers
27913 · Content-Type – .INDENT 2.0
27915 · application/json
27917 · Accept – .INDENT 2.0
27920 · application/json
27922 Request JSON Object
27925 · queries – An array of query objects with fields for the param‐
27927 eters of each individual view query to be executed. The field
27928 names and their meaning are the same as the query parameters
27929 of a regular view request.
27930 Response Headers
27932 · Content-Type – .INDENT 2.0
27934 · application/json
27936 · ETag – Response signature
27939 · Transfer-Encoding – chunked
27941 Response JSON Object
27943 · results (array) – An array of result objects - one for each
27945 query. Each result object contains the same fields as the
27946 response to a regular view request.
27947 Status Codes
27949 · 200 OK – Request completed successfully
27951 · 400 Bad Request – Invalid request
27953 · 401 Unauthorized – Read permission required
27955 · 404 Not Found – Specified database, design document or view is
27957 missing
27958 · 500 Internal Server Error – View function execution error
27960
27962 POST /recipes/_design/recipes/_view/by_title/queries HTTP/1.1
27964 Content-Type: application/json
27965 Accept: application/json
27966 Host: localhost:5984
27967 {
27969 "queries": [
27970 {
27971 "keys": [
27972 "meatballs",
27973 "spaghetti"
27974 ]
27975 },
27976 {
27977 "limit": 3,
27978 "skip": 2
27979 }
27980 ]
27981 }
27982 Response:
27984 HTTP/1.1 200 OK
27986 Cache-Control: must-revalidate
27987 Content-Type: application/json
27988 Date: Wed, 20 Dec 2016 11:17:07 GMT
27989 ETag: "1H8RGBCK3ABY6ACDM7ZSC30QK"
27990 Server: CouchDB (Erlang/OTP)
27991 Transfer-Encoding: chunked
27992 {
27994 "results" : [
27995 {
27996 "offset": 0,
27997 "rows": [
27998 {
27999 "id": "SpaghettiWithMeatballs",
28000 "key": "meatballs",
28001 "value": 1
28002 },
28003 {
28004 "id": "SpaghettiWithMeatballs",
28005 "key": "spaghetti",
28006 "value": 1
28007 },
28008 {
28009 "id": "SpaghettiWithMeatballs",
28010 "key": "tomato sauce",
28011 "value": 1
28012 }
28013 ],
28014 "total_rows": 3
28015 },
28016 {
28017 "offset" : 2,
28018 "rows" : [
28019 {
28020 "id" : "Adukiandorangecasserole-microwave",
28021 "key" : "Aduki and orange casserole - microwave",
28022 "value" : [
28023 null,
28024 "Aduki and orange casserole - microwave"
28025 ]
28026 },
28027 {
28028 "id" : "Aioli-garlicmayonnaise",
28029 "key" : "Aioli - garlic mayonnaise",
28030 "value" : [
28031 null,
28032 "Aioli - garlic mayonnaise"
28033 ]
28034 },
28035 {
28036 "id" : "Alabamapeanutchicken",
28037 "key" : "Alabama peanut chicken",
28038 "value" : [
28039 null,
28040 "Alabama peanut chicken"
28041 ]
28042 }
28043 ],
28044 "total_rows" : 2667
28045 }
28046 ]
28047 }
28048 /db/_design/design-doc/_search/index-name
28050 WARNING:
28051 Search endpoints require a running search plugin connected to each
28052 cluster node. See Search Plugin Installation for details.
28053 New in version 3.0.
28055 GET /{db}/_design/{ddoc}/_search/{index}
28058 Executes a search request against the named index in the speci‐
28059 fied design document.
28060 Parameters
28062 · db – Database name
28064 · ddoc – Design document name
28066 · index – Search index name
28068 Request Headers
28070 · Accept – .INDENT 2.0
28072 · application/json
28074 · text/plain
28076 Query Parameters
28079 · bookmark (string) – A bookmark received from a previous
28081 search. This parameter enables paging through the results. If
28082 there are no more results after the bookmark, you get a
28083 response with an empty rows array and the same bookmark, con‐
28084 firming the end of the result list.
28085 · counts (json) – An array of names of string fields for which
28087 counts are requested. The response contains counts for each
28088 unique value of this field name among the documents that match
28089 the search query. Faceting must be enabled for this parameter
28090 to function.
28091 · drilldown (json) – This field can be used several times. Each
28093 use defines a pair with a field name and a value. The search
28094 matches only documents containing the value that was provided
28095 in the named field. It differs from using "fieldname:value" in
28096 the q parameter only in that the values are not analyzed.
28097 Faceting must be enabled for this parameter to function.
28098 · group_field (string) – Field by which to group search matches.
28100 :query number group_limit: Maximum group count. This field can
28101 be used only if group_field is specified.
28102 · group_sort (json) – This field defines the order of the groups
28104 in a search that uses group_field. The default sort order is
28105 relevance.
28106 · highlight_fields (json) – Specifies which fields to highlight.
28108 If specified, the result object contains a highlights field
28109 with an entry for each specified field.
28110 · highlight_pre_tag (string) – A string that is inserted before
28112 the highlighted word in the highlights output.
28113 · highlight_post_tag (string) – A string that is inserted after
28115 the highlighted word in the highlights output.
28116 · highlight_number (number) – Number of fragments that are
28118 returned in highlights. If the search term occurs less often
28119 than the number of fragments that are specified, longer frag‐
28120 ments are returned.
28121 · highlight_size (number) – Number of characters in each frag‐
28123 ment for highlights.
28124 · include_docs (boolean) – Include the full content of the docu‐
28126 ments in the response.
28127 · include_fields (json) – A JSON array of field names to include
28129 in search results. Any fields that are included must be
28130 indexed with the store:true option.
28131 · limit (number) – Limit the number of the returned documents to
28133 the specified number. For a grouped search, this parameter
28134 limits the number of documents per group.
28135 · q (string) – Alias for query.
28137 · query (string) – Required. The Lucene query string.
28139 · ranges (json) – This field defines ranges for faceted, numeric
28141 search fields. The value is a JSON object where the fields
28142 names are faceted numeric search fields, and the values of the
28143 fields are JSON objects. The field names of the JSON objects
28144 are names for ranges. The values are strings that describe the
28145 range, for example “[0 TO 10]”.
28146 · sort (json) – Specifies the sort order of the results. In a
28148 grouped search (when group_field is used), this parameter
28149 specifies the sort order within a group. The default sort
28150 order is relevance. A JSON string of the form "field‐
28151 name<type>" or -fieldname<type> for descending order, where
28152 fieldname is the name of a string or number field, and type is
28153 either a number, a string, or a JSON array of strings. The
28154 type part is optional, and defaults to number. Some examples
28155 are "foo", "-foo", "bar<string>", "-foo<number>" and
28156 ["-foo<number>", "bar<string>"]. String fields that are used
28157 for sorting must not be analyzed fields. Fields that are used
28158 for sorting must be indexed by the same indexer that is used
28159 for the search query.
28160 · stale (string) – Set to ok to allow the use of an out-of-date
28162 index.
28163 Response Headers
28165 · Content-Type – .INDENT 2.0
28167 · application/json
28169 · text/plain; charset=utf-8
28171 · ETag – Response signature
28174 · Transfer-Encoding – chunked
28176 Response JSON Object
28178 · rows (array) – Array of view row objects. By default the
28180 information returned contains only the document ID and revi‐
28181 sion.
28182 · total_rows (number) – Number of documents in the data‐
28184 base/view.
28185 · bookmark (string) – Opaque identifier to enable pagination.
28187 Status Codes
28189 · 200 OK – Request completed successfully
28191 · 400 Bad Request – Invalid request
28193 · 401 Unauthorized – Read permission required
28195 · 404 Not Found – Specified database, design document or view is
28197 missed
28198
28200 You must enable faceting before you can use the counts, drilldown,
28201 and ranges parameters.
28202 NOTE:
28204 Faceting and grouping are not supported on partitioned searches, so
28205 the following query parameters should not be used on those requests:
28206 counts, drilldown, ranges, and group_field, group_limit,
28207 group_sort``.
28208 NOTE:
28210 Do not combine the bookmark and stale options. These options con‐
28211 strain the choice of shard replicas to use for the response. When
28212 used together, the options might cause problems when contact is
28213 attempted with replicas that are slow or not available.
28214 SEE ALSO:
28216 For more information about how search works, see the Search User
28217 Guide.
28218 /db/_design/design-doc/_search_info/index-name
28220 WARNING:
28221 Search endpoints require a running search plugin connected to each
28222 cluster node. See Search Plugin Installation for details.
28223 New in version 3.0.
28225 GET /{db}/_design/{ddoc}/_search_info/{index}
28228 Parameters
28230 · db – Database name
28232 · ddoc – Design document name
28234 · index – Search index name
28236 Status Codes
28238 · 200 OK – Request completed successfully
28240 · 400 Bad Request – Request body is wrong (malformed or
28242 missing one of the mandatory fields)
28243 · 500 Internal Server Error – A server error (or other
28245 kind of error) occurred
28246 Request:
28248 GET /recipes/_design/cookbook/_search_info/ingredients HTTP/1.1
28250 Accept: application/json
28251 Host: localhost:5984
28252 Response:
28254 {
28256 "name": "_design/cookbook/ingredients",
28257 "search_index": {
28258 "pending_seq": 7125496,
28259 "doc_del_count": 129180,
28260 "doc_count": 1066173,
28261 "disk_size": 728305827,
28262 "committed_seq": 7125496
28263 }
28264 }
28265 /db/_design/design-doc/_show/show-name
28267 WARNING:
28268 Show functions are deprecated in CouchDB 3.0, and will be removed in
28269 CouchDB 4.0.
28270 GET /{db}/_design/{ddoc}/_show/{func}
28272 POST /{db}/_design/{ddoc}/_show/{func}
28274 Applies show function for null document.
28275 The request and response parameters are depended upon function
28277 implementation.
28278 Parameters
28280 · db – Database name
28282 · ddoc – Design document name
28284 · func – Show function name
28286 Response Headers
28288 · ETag – Response signature
28290 Query Parameters
28292 · format (string) – Format of the returned response.
28294 Used by provides() function
28295 Status Codes
28297 · 200 OK – Request completed successfully
28299 · 500 Internal Server Error – Query server error
28301 Function:
28303 function(doc, req) {
28305 if (!doc) {
28306 return {body: "no doc"}
28307 } else {
28308 return {body: doc.description}
28309 }
28310 }
28311 Request:
28313 GET /recipes/_design/recipe/_show/description HTTP/1.1
28315 Accept: application/json
28316 Host: localhost:5984
28317 Response:
28319 HTTP/1.1 200 OK
28321 Content-Length: 6
28322 Content-Type: text/html; charset=utf-8
28323 Date: Wed, 21 Aug 2013 12:34:07 GMT
28324 Etag: "7Z2TO7FPEMZ0F4GH0RJCRIOAU"
28325 Server: CouchDB (Erlang/OTP)
28326 Vary: Accept
28327 no doc
28329 /db/_design/design-doc/_show/show-name/doc-id
28331 WARNING:
28332 Show functions are deprecated in CouchDB 3.0, and will be removed in
28333 CouchDB 4.0.
28334 GET /{db}/_design/{ddoc}/_show/{func}/{docid}
28336 POST /{db}/_design/{ddoc}/_show/{func}/{docid}
28338 Applies show function for the specified document.
28339 The request and response parameters are depended upon function
28341 implementation.
28342 Parameters
28344 · db – Database name
28346 · ddoc – Design document name
28348 · func – Show function name
28350 · docid – Document ID
28352 Response Headers
28354 · ETag – Response signature
28356 Query Parameters
28358 · format (string) – Format of the returned response.
28360 Used by provides() function
28361 Status Codes
28363 · 200 OK – Request completed successfully
28365 · 500 Internal Server Error – Query server error
28367 Function:
28369 function(doc, req) {
28371 if (!doc) {
28372 return {body: "no doc"}
28373 } else {
28374 return {body: doc.description}
28375 }
28376 }
28377 Request:
28379 GET /recipes/_design/recipe/_show/description/SpaghettiWithMeatballs HTTP/1.1
28381 Accept: application/json
28382 Host: localhost:5984
28383 Response:
28385 HTTP/1.1 200 OK
28387 Content-Length: 88
28388 Content-Type: text/html; charset=utf-8
28389 Date: Wed, 21 Aug 2013 12:38:08 GMT
28390 Etag: "8IEBO8103EI98HDZL5Z4I1T0C"
28391 Server: CouchDB (Erlang/OTP)
28392 Vary: Accept
28393 An Italian-American dish that usually consists of spaghetti, tomato sauce and meatballs.
28395 /db/_design/design-doc/_list/list-name/view-name
28397 WARNING:
28398 List functions are deprecated in CouchDB 3.0, and will be removed in
28399 CouchDB 4.0.
28400 GET /{db}/_design/{ddoc}/_list/{func}/{view}
28402 POST /{db}/_design/{ddoc}/_list/{func}/{view}
28404 Applies list function for the view function from the same design
28405 document.
28406 The request and response parameters are depended upon function
28408 implementation.
28409 Parameters
28411 · db – Database name
28413 · ddoc – Design document name
28415 · func – List function name
28417 · view – View function name
28419 Response Headers
28421 · ETag – Response signature
28423 · Transfer-Encoding – chunked
28425 Query Parameters
28427 · format (string) – Format of the returned response.
28429 Used by provides() function
28430 Status Codes
28432 · 200 OK – Request completed successfully
28434 · 500 Internal Server Error – Query server error
28436 Function:
28438 function(head, req) {
28440 var row = getRow();
28441 if (!row){
28442 return 'no ingredients'
28443 }
28444 send(row.key);
28445 while(row=getRow()){
28446 send(', ' + row.key);
28447 }
28448 }
28449 Request:
28451 GET /recipes/_design/recipe/_list/ingredients/by_name HTTP/1.1
28453 Accept: text/plain
28454 Host: localhost:5984
28455 Response:
28457 HTTP/1.1 200 OK
28459 Content-Type: text/plain; charset=utf-8
28460 Date: Wed, 21 Aug 2013 12:49:15 GMT
28461 Etag: "D52L2M1TKQYDD1Y8MEYJR8C84"
28462 Server: CouchDB (Erlang/OTP)
28463 Transfer-Encoding: chunked
28464 Vary: Accept
28465 meatballs, spaghetti, tomato sauce
28467 /db/_design/design-doc/_list/list-name/other-ddoc/view-name
28469 WARNING:
28470 List functions are deprecated in CouchDB 3.0, and will be removed in
28471 CouchDB 4.0.
28472 GET /{db}/_design/{ddoc}/_list/{func}/{other-ddoc}/{view}
28474 POST /{db}/_design/{ddoc}/_list/{func}/{other-ddoc}/{view}
28476 Applies list function for the view function from the other
28477 design document.
28478 The request and response parameters are depended upon function
28480 implementation.
28481 Parameters
28483 · db – Database name
28485 · ddoc – Design document name
28487 · func – List function name
28489 · other-ddoc – Other design document name that holds view
28491 function
28492 · view – View function name
28494 Response Headers
28496 · ETag – Response signature
28498 · Transfer-Encoding – chunked
28500 Query Parameters
28502 · format (string) – Format of the returned response.
28504 Used by provides() function
28505 Status Codes
28507 · 200 OK – Request completed successfully
28509 · 500 Internal Server Error – Query server error
28511 Function:
28513 function(head, req) {
28515 var row = getRow();
28516 if (!row){
28517 return 'no ingredients'
28518 }
28519 send(row.key);
28520 while(row=getRow()){
28521 send(', ' + row.key);
28522 }
28523 }
28524 Request:
28526 GET /recipes/_design/ingredient/_list/ingredients/recipe/by_ingredient?key="spaghetti" HTTP/1.1
28528 Accept: text/plain
28529 Host: localhost:5984
28530 Response:
28532 HTTP/1.1 200 OK
28534 Content-Type: text/plain; charset=utf-8
28535 Date: Wed, 21 Aug 2013 12:49:15 GMT
28536 Etag: "5L0975X493R0FB5Z3043POZHD"
28537 Server: CouchDB (Erlang/OTP)
28538 Transfer-Encoding: chunked
28539 Vary: Accept
28540 spaghetti
28542 /db/_design/design-doc/_update/update-name
28544 POST /{db}/_design/{ddoc}/_update/{func}
28545 Executes update function on server side for null document.
28546 Parameters
28548 · db – Database name
28550 · ddoc – Design document name
28552 · func – Update function name
28554 Response Headers
28556 · X-Couch-Id – Created/updated document’s ID
28558 · X-Couch-Update-Newrev – Created/updated document’s
28560 revision
28561 Status Codes
28563 · 200 OK – No document was created or updated
28565 · 201 Created – Document was created or updated
28567 · 500 Internal Server Error – Query server error
28569 Function:
28571 function(doc, req) {
28573 if (!doc){
28574 return [null, {'code': 400,
28575 'json': {'error': 'missed',
28576 'reason': 'no document to update'}}]
28577 } else {
28578 doc.ingredients.push(req.body);
28579 return [doc, {'json': {'status': 'ok'}}];
28580 }
28581 }
28582 Request:
28584 POST /recipes/_design/recipe/_update/ingredients HTTP/1.1
28586 Accept: application/json
28587 Content-Length: 10
28588 Content-Type: application/json
28589 Host: localhost:5984
28590 "something"
28592 Response:
28594 HTTP/1.1 404 Object Not Found
28596 Cache-Control: must-revalidate
28597 Content-Length: 52
28598 Content-Type: application/json
28599 Date: Wed, 21 Aug 2013 14:00:58 GMT
28600 Server: CouchDB (Erlang/OTP)
28601 {
28603 "error": "missed",
28604 "reason": "no document to update"
28605 }
28606 /db/_design/design-doc/_update/update-name/doc-id
28608 PUT /{db}/_design/{ddoc}/_update/{func}/{docid}
28609 Executes update function on server side for the specified docu‐
28610 ment.
28611 Parameters
28613 · db – Database name
28615 · ddoc – Design document name
28617 · func – Update function name
28619 · docid – Document ID
28621 Response Headers
28623 · X-Couch-Id – Created/updated document’s ID
28625 · X-Couch-Update-Newrev – Created/updated document’s
28627 revision
28628 Status Codes
28630 · 200 OK – No document was created or updated
28632 · 201 Created – Document was created or updated
28634 · 500 Internal Server Error – Query server error
28636 Function:
28638 function(doc, req) {
28640 if (!doc){
28641 return [null, {'code': 400,
28642 'json': {'error': 'missed',
28643 'reason': 'no document to update'}}]
28644 } else {
28645 doc.ingredients.push(req.body);
28646 return [doc, {'json': {'status': 'ok'}}];
28647 }
28648 }
28649 Request:
28651 POST /recipes/_design/recipe/_update/ingredients/SpaghettiWithMeatballs HTTP/1.1
28653 Accept: application/json
28654 Content-Length: 5
28655 Content-Type: application/json
28656 Host: localhost:5984
28657 "love"
28659 Response:
28661 HTTP/1.1 201 Created
28663 Cache-Control: must-revalidate
28664 Content-Length: 16
28665 Content-Type: application/json
28666 Date: Wed, 21 Aug 2013 14:11:34 GMT
28667 Server: CouchDB (Erlang/OTP)
28668 X-Couch-Id: SpaghettiWithMeatballs
28669 X-Couch-Update-NewRev: 12-a5e099df5720988dae90c8b664496baf
28670 {
28672 "status": "ok"
28673 }
28674 /db/_design/design-doc/_rewrite/path
28676 WARNING:
28677 Rewrites are deprecated in CouchDB 3.0, and will be removed in
28678 CouchDB 4.0.
28679 ANY /{db}/_design/{ddoc}/_rewrite/{path}
28681 Rewrites the specified path by rules defined in the specified
28682 design document. The rewrite rules are defined by the rewrites
28683 field of the design document. The rewrites field can either be a
28684 string containing the a rewrite function or an array of rule
28685 definitions.
28686 Using a stringified function for rewrites
28688 New in version 2.0: When the rewrites field is a stringified function,
28689 the query server is used to pre-process and route requests.
28690 The function takes a request2_object.
28692 The return value of the function will cause the server to rewrite the
28694 request to a new location or immediately return a response.
28695 To rewrite the request, return an object containing the following prop‐
28697 erties:
28698 · path (string): Rewritten path.
28700 · query (array): Rewritten query. If omitted, the original query keys
28702 are used.
28703 · headers (object): Rewritten headers. If omitted, the original request
28705 headers are used.
28706 · method (string): HTTP method of rewritten request ("GET", "POST",
28708 etc). If omitted, the original request method is used.
28709 · body (string): Body for "POST"/"PUT" requests. If omitted, the origi‐
28711 nal request body is used.
28712 To immediately respond to the request, return an object containing the
28714 following properties:
28715 · code (number): Returned HTTP status code (200, 404, etc).
28717 · body (string): Body of the response to user.
28719 Example A. Restricting access.
28721 function(req2) {
28723 var path = req2.path.slice(4),
28724 isWrite = /^(put|post|delete)$/i.test(req2.method),
28725 isFinance = req2.userCtx.roles.indexOf("finance") > -1;
28726 if (path[0] == "finance" && isWrite && !isFinance) {
28727 // Deny writes to DB "finance" for users
28728 // having no "finance" role
28729 return {
28730 code: 403,
28731 body: JSON.stringify({
28732 error: "forbidden".
28733 reason: "You are not allowed to modify docs in this DB"
28734 })
28735 };
28736 }
28737 // Pass through all other requests
28738 return { path: "../../../" + path.join("/") };
28739 }
28740 Example B. Different replies for JSON and HTML requests.
28742 function(req2) {
28744 var path = req2.path.slice(4),
28745 h = headers,
28746 wantsJson = (h.Accept || "").indexOf("application/json") > -1,
28747 reply = {};
28748 if (!wantsJson) {
28749 // Here we should prepare reply object
28750 // for plain HTML pages
28751 } else {
28752 // Pass through JSON requests
28753 reply.path = "../../../"+path.join("/");
28754 }
28755 return reply;
28756 }
28757 Using an array of rules for rewrites
28760 When the rewrites field is an array of rule objects, the server will
28761 rewrite the request based on the first matching rule in the array.
28762 Each rule in the array is an object with the following fields:
28764 · method (string): HTTP request method to bind the request method to
28766 the rule. If omitted, uses "*", which matches all methods.
28767 · from (string): The pattern used to compare against the URL and
28769 define dynamic variables.
28770 · to (string): The path to rewrite the URL to. It can contain vari‐
28772 ables depending on binding variables discovered during pattern
28773 matching and query args (URL args and from the query member).
28774 · query (object): Query args passed to the rewritten URL. They may
28776 contain dynamic variables.
28777 The to and from paths may contains string patterns with leading : or
28779 * characters to define dynamic variables in the match.
28780 The first rule in the rewrites array that matches the incoming
28782 request is used to define the rewrite. To match the incoming
28783 request, the rule’s method must match the request’s HTTP method and
28784 the rule’s from must match the request’s path using the following
28785 pattern matching logic.
28786 · The from pattern and URL are first split on / to get a list of
28788 tokens. For example, if from field is /somepath/:var/* and the URL
28789 is /somepath/a/b/c, the tokens are somepath, :var, and * for the
28790 from pattern and somepath, a, b, and c for the URL.
28791 · Each token starting with : in the pattern will match the corre‐
28793 sponding token in the URL and define a new dynamic variable whose
28794 name is the remaining string after the : and value is the token
28795 from the URL. In this example, the :var token will match b and set
28796 var = a.
28797 · The star token * in the pattern will match any number of tokens in
28799 the URL and must be the last token in the pattern. It will define
28800 a dynamic variable with the remaining tokens. In this example, the
28801 * token will match the b and c tokens and set * = b/c.
28802 · The remaining tokens must match exactly for the pattern to be con‐
28804 sidered a match. In this example, somepath in the pattern matches
28805 somepath in the URL and all tokens in the URL have matched, caus‐
28806 ing this rule to be a match.
28807 Once a rule is found, the request URL is rewritten using the to and
28809 query fields. Dynamic variables are substituted into the : and *
28810 variables in these fields to produce the final URL.
28811 If no rule matches, a 404 Not Found response is returned.
28813 Examples:
28815 ┌────────────────────────────────┬──────────┬──────────────────┬────────┐
28817 │Rule │ URL │ Rewrite to │ Tokens │
28818 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28819 │ │ /a │ /some │ │
28820 │ {“from”: │ │ │ │
28821 │ “/a”, │ │ │ │
28822 │ “to”: │ │ │ │
28823 │ “/some”} │ │ │ │
28824 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28825 │ │ /a/b/c │ /some/b/c │ │
28826 │ {“from”: │ │ │ │
28827 │ “/a/*”, │ │ │ │
28828 │ “to”: │ │ │ │
28829 │ “/some/*} │ │ │ │
28830 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28831 │ │ /a/b?k=v │ /some?k=v │ k=v │
28832 │ {“from”: “/a/b”, │ │ │ │
28833 │ “to”: │ │ │ │
28834 │ “/some”} │ │ │ │
28835 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28836 │ │ /a/b │ /some/b?var=b │ var=b │
28837 │ {“from”: “/a/b”, │ │ │ │
28838 │ “to”: │ │ │ │
28839 │ “/some/:var”} │ │ │ │
28840 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28841 │ │ /a/b/c │ /some/b/c?foo=b │ foo=b │
28842 │ {“from”: “/a/:foo/”, │ │ │ │
28843 │ “to”: │ │ │ │
28844 │ “/some/:foo/”} │ │ │ │
28845 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28846 │ │ /a/b │ /some/?k=b&foo=b │ foo=b │
28847 │ {“from”: “/a/:foo”, │ │ │ │
28848 │ “to”: “/some”, │ │ │ │
28849 │ “query”: { │ │ │ │
28850 │ “k”: “:foo” }} │ │ │ │
28851 ├────────────────────────────────┼──────────┼──────────────────┼────────┤
28852 │ │ /a?foo=b │ /some/?b&foo=b │ foo=b │
28853 │ {“from”: “/a”, │ │ │ │
28854 │ “to”: │ │ │ │
28855 │ “/some/:foo”} │ │ │ │
28856 └────────────────────────────────┴──────────┴──────────────────┴────────┘
28857 Request method, header, query parameters, request payload and
28859 response body are dependent on the endpoint to which the URL will be
28860 rewritten.
28861 param db
28863 Database name
28864 param ddoc
28866 Design document name
28867 param path
28869 URL path to rewrite
28870 Partitioned Databases
28872 Partitioned databases allow for data colocation in a cluster, which
28873 provides significant performance improvements for queries constrained
28874 to a single partition.
28875 See the guide for getting started with partitioned databases
28877 /db/_partition/partition
28879 GET /{db}/_partition/{partition}
28880 This endpoint returns information describing the provided parti‐
28881 tion. It includes document and deleted document counts along
28882 with external and active data sizes.
28883 Status Codes
28885 · 200 OK – Request completed successfully
28887 Request:
28889 GET /db/_partition/foo HTTP/1.1
28891 Accept: application/json
28892 Host: localhost:5984
28893 Response:
28895 HTTP/1.1 200 OK
28897 Cache-Control: must-revalidate
28898 Content-Length: 119
28899 Content-Type: application/json
28900 Date: Thu, 24 Jan 2019 17:19:59 GMT
28901 Server: CouchDB/2.3.0-a1e11cea9 (Erlang OTP/21)
28902 {
28904 "db_name": "my_new_db",
28905 "doc_count": 1,
28906 "doc_del_count": 0,
28907 "partition": "sensor-260",
28908 "sizes": {
28909 "active": 244,
28910 "external": 347
28911 }
28912 }
28913 /db/_partition/partition/_all_docs
28915 GET /{db}/_partition/{partition}/_all_docs
28916 Parameters
28918 · db – Database name
28920 · partition – Partition name
28922 This endpoint is a convenience endpoint for automatically set‐
28924 ting bounds on the provided partition range. Similar results can
28925 be had by using the global /db/_all_docs endpoint with appropri‐
28926 ately configured values for start_key and end_key.
28927 Refer to the view endpoint documentation for a complete descrip‐
28929 tion of the available query parameters and the format of the
28930 returned data.
28931 Request:
28933 GET /db/_partition/sensor-260/_all_docs HTTP/1.1
28935 Accept: application/json
28936 Host: localhost:5984
28937 Response:
28939 HTTP/1.1 200 OK
28941 Cache-Control: must-revalidate
28942 Content-Type: application/json
28943 Date: Sat, 10 Aug 2013 16:22:56 GMT
28944 ETag: "1W2DJUZFZSZD9K78UFA3GZWB4"
28945 Server: CouchDB (Erlang/OTP)
28946 Transfer-Encoding: chunked
28947 {
28949 "offset": 0,
28950 "rows": [
28951 {
28952 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
28953 "key": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
28954 "value": {
28955 "rev": "1-05ed6f7abf84250e213fcb847387f6f5"
28956 }
28957 }
28958 ],
28959 "total_rows": 1
28960 }
28961 /db/_partition/partition/_design/design-doc/_view/view-name
28963 GET /{db}/_partition/{partition}/_design/{ddoc}/_view/{view}
28964 Parameters
28966 · db – Database name
28968 · partition – Partition name
28970 · ddoc – Design document id
28972 · view – View name
28974 This endpoint is responsible for executing a partitioned query.
28976 The returned view result will only contain rows with the speci‐
28977 fied partition name.
28978 Refer to the view endpoint documentation for a complete descrip‐
28980 tion of the available query parameters and the format of the
28981 returned data.
28982 GET /db/_partition/sensor-260/_design/sensor-readings/_view/by_sensor HTTP/1.1
28984 Accept: application/json
28985 Host: localhost:5984
28986 Response:
28988 HTTP/1.1 200 OK
28990 Cache-Control: must-revalidate
28991 Content-Type: application/json
28992 Date: Wed, 21 Aug 2013 09:12:06 GMT
28993 ETag: "2FOLSBSW4O6WB798XU4AQYA9B"
28994 Server: CouchDB (Erlang/OTP)
28995 Transfer-Encoding: chunked
28996 {
28998 "offset": 0,
28999 "rows": [
29000 {
29001 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29002 "key": [
29003 "sensor-260",
29004 "0"
29005 ],
29006 "value": null
29007 },
29008 {
29009 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29010 "key": [
29011 "sensor-260",
29012 "1"
29013 ],
29014 "value": null
29015 },
29016 {
29017 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29018 "key": [
29019 "sensor-260",
29020 "2"
29021 ],
29022 "value": null
29023 },
29024 {
29025 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
29026 "key": [
29027 "sensor-260",
29028 "3"
29029 ],
29030 "value": null
29031 }
29032 ],
29033 "total_rows": 4
29034 }
29035 /db/_partition/partition_id/_find
29037 GET /{db}/_partition/{partition_id}/_find
29038 Parameters
29040 · db – Database name
29042 · id (partition) – Name of the partition to query
29044 This endpoint is responsible for finding a partition query by
29046 its ID. The returned view result will only contain rows with
29047 the specified partition id.
29048 Refer to the find endpoint documentation for a complete descrip‐
29050 tion of the available parameters and the format of the returned
29051 data.
29052 /db/_partition/partition_id/_explain
29054 GET /{db}/_partition/{partition_id}/_explain
29055 Parameters
29057 · db – Database name
29059 Partition id
29061 Name of the partition to query
29062 This endpoint shows which index is being used by the query.
29064 Refer to the explain endpoint documentation for a complete
29066 description of the available parameters and the format of the
29067 returned data.
29068 Local (non-replicating) Documents
29070 The Local (non-replicating) document interface allows you to create
29071 local documents that are not replicated to other databases. These docu‐
29072 ments can be used to hold configuration or other information that is
29073 required specifically on the local CouchDB instance.
29074 Local documents have the following limitations:
29076 · Local documents are not replicated to other databases.
29078 · Local documents are not output by views, or the api/db/all_docs view.
29080 From CouchDB 2.0, Local documents can be listed by using the
29082 /db/_local_docs endpoint.
29083 Local documents can be used when you want to store configuration or
29085 other information for the current (local) instance of a given database.
29086 A list of the available methods and URL paths are provided below:
29088 ┌──────────┬─────────────────┬─────────────────────┐
29090 │Method │ Path │ Description │
29091 ├──────────┼─────────────────┼─────────────────────┤
29092 │GET, POST │ /db/_local_docs │ Returns a list of │
29093 │ │ │ all the non-repli‐ │
29094 │ │ │ cated documents in │
29095 │ │ │ the database │
29096 ├──────────┼─────────────────┼─────────────────────┤
29097 │GET │ /db/_local/id │ Returns the latest │
29098 │ │ │ revision of the │
29099 │ │ │ non-replicated doc‐ │
29100 │ │ │ ument │
29101 ├──────────┼─────────────────┼─────────────────────┤
29102 │PUT │ /db/_local/id │ Inserts a new ver‐ │
29103 │ │ │ sion of the │
29104 │ │ │ non-replicated doc‐ │
29105 │ │ │ ument │
29106 ├──────────┼─────────────────┼─────────────────────┤
29107 │DELETE │ /db/_local/id │ Deletes the │
29108 │ │ │ non-replicated doc‐ │
29109 │ │ │ ument │
29110 ├──────────┼─────────────────┼─────────────────────┤
29111 │COPY │ /db/_local/id │ Copies the │
29112 │ │ │ non-replicated doc‐ │
29113 │ │ │ ument │
29114 └──────────┴─────────────────┴─────────────────────┘
29115 /db/_local_docs
29117 GET /{db}/_local_docs
29118 Returns a JSON structure of all of the local documents in a
29119 given database. The information is returned as a JSON structure
29120 containing meta information about the return structure, includ‐
29121 ing a list of all local documents and basic contents, consisting
29122 the ID, revision and key. The key is the from the local docu‐
29123 ment’s _id.
29124 Parameters
29126 · db – Database name
29128 Request Headers
29130 · Accept – .INDENT 2.0
29132 · application/json
29134 · text/plain
29136 Query Parameters
29139 · conflicts (boolean) – Includes conflicts information in
29141 response. Ignored if include_docs isn’t true. Default is
29142 false.
29143 · descending (boolean) – Return the local documents in descend‐
29145 ing by key order. Default is false.
29146 · endkey (string) – Stop returning records when the specified
29148 key is reached. Optional.
29149 · end_key (string) – Alias for endkey param.
29151 · endkey_docid (string) – Stop returning records when the speci‐
29153 fied local document ID is reached. Optional.
29154 · end_key_doc_id (string) – Alias for endkey_docid param.
29156 · include_docs (boolean) – Include the full content of the local
29158 documents in the return. Default is false.
29159 · inclusive_end (boolean) – Specifies whether the specified end
29161 key should be included in the result. Default is true.
29162 · key (string) – Return only local documents that match the
29164 specified key. Optional.
29165 · keys (string) – Return only local documents that match the
29167 specified keys. Optional.
29168 · limit (number) – Limit the number of the returned local docu‐
29170 ments to the specified number. Optional.
29171 · skip (number) – Skip this number of records before starting to
29173 return the results. Default is 0.
29174 · startkey (string) – Return records starting with the specified
29176 key. Optional.
29177 · start_key (string) – Alias for startkey param.
29179 · startkey_docid (string) – Return records starting with the
29181 specified local document ID. Optional.
29182 · start_key_doc_id (string) – Alias for startkey_docid param.
29184 · update_seq (boolean) – Response includes an update_seq value
29186 indicating which sequence id of the underlying database the
29187 view reflects. Default is false.
29188 Response Headers
29190 · Content-Type – .INDENT 2.0
29192 · application/json
29194 · text/plain; charset=utf-8
29196 Response JSON Object
29199 · offset (number) – Offset where the local document list started
29201 · rows (array) – Array of view row objects. By default the
29203 information returned contains only the local document ID and
29204 revision.
29205 · total_rows (number) – Number of local documents in the data‐
29207 base. Note that this is not the number of rows returned in the
29208 actual query.
29209 · update_seq (number) – Current update sequence for the database
29211 Status Codes
29213 · 200 OK – Request completed successfully
29215
29217 GET /db/_local_docs HTTP/1.1
29219 Accept: application/json
29220 Host: localhost:5984
29221 Response:
29223 HTTP/1.1 200 OK
29225 Cache-Control: must-revalidate
29226 Content-Type: application/json
29227 Date: Sat, 23 Dec 2017 16:22:56 GMT
29228 Server: CouchDB (Erlang/OTP)
29229 Transfer-Encoding: chunked
29230 {
29232 "offset": null,
29233 "rows": [
29234 {
29235 "id": "_local/localdoc01",
29236 "key": "_local/localdoc01",
29237 "value": {
29238 "rev": "0-1"
29239 }
29240 },
29241 {
29242 "id": "_local/localdoc02",
29243 "key": "_local/localdoc02",
29244 "value": {
29245 "rev": "0-1"
29246 }
29247 },
29248 {
29249 "id": "_local/localdoc03",
29250 "key": "_local/localdoc03",
29251 "value": {
29252 "rev": "0-1"
29253 }
29254 },
29255 {
29256 "id": "_local/localdoc04",
29257 "key": "_local/localdoc04",
29258 "value": {
29259 "rev": "0-1"
29260 }
29261 },
29262 {
29263 "id": "_local/localdoc05",
29264 "key": "_local/localdoc05",
29265 "value": {
29266 "rev": "0-1"
29267 }
29268 }
29269 ],
29270 "total_rows": null
29271 }
29272 POST /{db}/_local_docs
29274 POST _local_docs functionality supports identical parameters and
29275 behavior as specified in the GET /{db}/_local_docs API but
29276 allows for the query string parameters to be supplied as keys in
29277 a JSON object in the body of the POST request.
29278 Request:
29280 POST /db/_local_docs HTTP/1.1
29282 Accept: application/json
29283 Content-Length: 70
29284 Content-Type: application/json
29285 Host: localhost:5984
29286 {
29288 "keys" : [
29289 "_local/localdoc02",
29290 "_local/localdoc05"
29291 ]
29292 }
29293 The returned JSON is the all documents structure, but with only
29295 the selected keys in the output:
29296 {
29298 "total_rows" : null,
29299 "rows" : [
29300 {
29301 "value" : {
29302 "rev" : "0-1"
29303 },
29304 "id" : "_local/localdoc02",
29305 "key" : "_local/localdoc02"
29306 },
29307 {
29308 "value" : {
29309 "rev" : "0-1"
29310 },
29311 "id" : "_local/localdoc05",
29312 "key" : "_local/localdoc05"
29313 }
29314 ],
29315 "offset" : null
29316 }
29317 /db/_local/id
29319 GET /{db}/_local/{docid}
29320 Gets the specified local document. The semantics are identical
29321 to accessing a standard document in the specified database,
29322 except that the document is not replicated. See GET
29323 /{db}/{docid}.
29324 PUT /{db}/_local/{docid}
29326 Stores the specified local document. The semantics are identical
29327 to storing a standard document in the specified database, except
29328 that the document is not replicated. See PUT /{db}/{docid}.
29329 DELETE /{db}/_local/{docid}
29331 Deletes the specified local document. The semantics are identi‐
29332 cal to deleting a standard document in the specified database,
29333 except that the document is not replicated. See DELETE
29334 /{db}/{docid}.
29335 COPY /{db}/_local/{docid}
29337 Copies the specified local document. The semantics are identical
29338 to copying a standard document in the specified database, except
29339 that the document is not replicated. See COPY /{db}/{docid}.
29340
29342 The following appendix provides a quick reference to all the JSON
29343 structures that you can supply to CouchDB, or get in return to
29344 requests.
29345 All Database Documents
29347 ┌──────────────────────┬────────────────────────────┐
29348 │Field │ Description │
29349 ├──────────────────────┼────────────────────────────┤
29350 │total_rows │ Number of documents in the │
29351 │ │ database/view │
29352 ├──────────────────────┼────────────────────────────┤
29353 │offset │ Offset where the document │
29354 │ │ list started │
29355 ├──────────────────────┼────────────────────────────┤
29356 │update_seq (optional) │ Current update sequence │
29357 │ │ for the database │
29358 ├──────────────────────┼────────────────────────────┤
29359 │rows [array] │ Array of document object │
29360 └──────────────────────┴────────────────────────────┘
29361 Bulk Document Response
29363 ┌─────────────┬────────────────────────────┐
29364 │Field │ Description │
29365 ├─────────────┼────────────────────────────┤
29366 │docs [array] │ Bulk Docs Returned Docu‐ │
29367 │ │ ments │
29368 ├─────────────┼────────────────────────────┤
29369 │id │ Document ID │
29370 ├─────────────┼────────────────────────────┤
29371 │error │ Error type │
29372 ├─────────────┼────────────────────────────┤
29373 │reason │ Error string with extended │
29374 │ │ reason │
29375 └─────────────┴────────────────────────────┘
29376 Bulk Documents
29378 ┌────────────────────┬────────────────────────────┐
29379 │Field │ Description │
29380 ├────────────────────┼────────────────────────────┤
29381 │docs [array] │ Bulk Documents Document │
29382 ├────────────────────┼────────────────────────────┤
29383 │_id (optional) │ Document ID │
29384 ├────────────────────┼────────────────────────────┤
29385 │_rev (optional) │ Revision ID (when updating │
29386 │ │ an existing document) │
29387 ├────────────────────┼────────────────────────────┤
29388 │_deleted (optional) │ Whether the document │
29389 │ │ should be deleted │
29390 └────────────────────┴────────────────────────────┘
29391 Changes information for a database
29393 ┌────────────────┬────────────────────────────┐
29394 │Field │ Description │
29395 ├────────────────┼────────────────────────────┤
29396 │last_seq │ Last update sequence │
29397 ├────────────────┼────────────────────────────┤
29398 │pending │ Count of remaining items │
29399 │ │ in the feed │
29400 ├────────────────┼────────────────────────────┤
29401 │results [array] │ Changes made to a database │
29402 ├────────────────┼────────────────────────────┤
29403 │seq │ Update sequence │
29404 ├────────────────┼────────────────────────────┤
29405 │id │ Document ID │
29406 ├────────────────┼────────────────────────────┤
29407 │changes [array] │ List of changes, │
29408 │ │ field-by-field, for this │
29409 │ │ document │
29410 └────────────────┴────────────────────────────┘
29411 CouchDB Document
29413 ┌────────────────┬────────────────────────────┐
29414 │Field │ Description │
29415 ├────────────────┼────────────────────────────┤
29416 │_id (optional) │ Document ID │
29417 ├────────────────┼────────────────────────────┤
29418 │_rev (optional) │ Revision ID (when updating │
29419 │ │ an existing document) │
29420 └────────────────┴────────────────────────────┘
29421 CouchDB Error Status
29423 ┌───────┬────────────────────────────┐
29424 │Field │ Description │
29425 ├───────┼────────────────────────────┤
29426 │id │ Document ID │
29427 ├───────┼────────────────────────────┤
29428 │error │ Error type │
29429 ├───────┼────────────────────────────┤
29430 │reason │ Error string with extended │
29431 │ │ reason │
29432 └───────┴────────────────────────────┘
29433 CouchDB database information object
29435 ┌─────────────────────┬────────────────────────────┐
29436 │Field │ Description │
29437 ├─────────────────────┼────────────────────────────┤
29438 │db_name │ The name of the database. │
29439 ├─────────────────────┼────────────────────────────┤
29440 │committed_update_seq │ The number of committed │
29441 │ │ updates. │
29442 ├─────────────────────┼────────────────────────────┤
29443 │doc_count │ The number of documents in │
29444 │ │ the database. │
29445 ├─────────────────────┼────────────────────────────┤
29446 │doc_del_count │ The number of deleted doc‐ │
29447 │ │ uments. │
29448 ├─────────────────────┼────────────────────────────┤
29449 │compact_running │ Set to true if the data‐ │
29450 │ │ base compaction routine is │
29451 │ │ operating on this data‐ │
29452 │ │ base. │
29453 └─────────────────────┴────────────────────────────┘
29454 │disk_format_version │ The version of the physi‐ │
29460 │ │ cal format used for the │
29461 │ │ data when it is stored on │
29462 │ │ hard disk. │
29463 ├─────────────────────┼────────────────────────────┤
29464 │disk_size │ Size in bytes of the data │
29465 │ │ as stored on disk. View │
29466 │ │ indexes are not included │
29467 │ │ in the calculation. │
29468 ├─────────────────────┼────────────────────────────┤
29469 │instance_start_time │ Timestamp indicating when │
29470 │ │ the database was opened, │
29471 │ │ expressed in microseconds │
29472 │ │ since the epoch. │
29473 ├─────────────────────┼────────────────────────────┤
29474 │purge_seq │ The number of purge opera‐ │
29475 │ │ tions on the database. │
29476 ├─────────────────────┼────────────────────────────┤
29477 │update_seq │ Current update sequence │
29478 │ │ for the database. │
29479 └─────────────────────┴────────────────────────────┘
29480 Design Document
29482 ┌──────────────────┬──────────────────────────┐
29483 │Field │ Description │
29484 ├──────────────────┼──────────────────────────┤
29485 │_id │ Design Document ID │
29486 ├──────────────────┼──────────────────────────┤
29487 │_rev │ Design Document Revision │
29488 ├──────────────────┼──────────────────────────┤
29489 │views │ View │
29490 ├──────────────────┼──────────────────────────┤
29491 │viewname │ View Definition │
29492 ├──────────────────┼──────────────────────────┤
29493 │map │ Map Function for View │
29494 ├──────────────────┼──────────────────────────┤
29495 │reduce (optional) │ Reduce Function for View │
29496 └──────────────────┴──────────────────────────┘
29497 Design Document Information
29499 ┌────────────────┬────────────────────────────┐
29500 │Field │ Description │
29501 ├────────────────┼────────────────────────────┤
29502 │name │ Name/ID of Design Document │
29503 ├────────────────┼────────────────────────────┤
29504 │view_index │ View Index │
29505 ├────────────────┼────────────────────────────┤
29506 │compact_running │ Indicates whether a com‐ │
29507 │ │ paction routine is cur‐ │
29508 │ │ rently running on the view │
29509 ├────────────────┼────────────────────────────┤
29510 │disk_size │ Size in bytes of the view │
29511 │ │ as stored on disk │
29512 ├────────────────┼────────────────────────────┤
29513 │language │ Language for the defined │
29514 │ │ views │
29515 ├────────────────┼────────────────────────────┤
29516 │purge_seq │ The purge sequence that │
29517 │ │ has been processed │
29518 ├────────────────┼────────────────────────────┤
29519 │signature │ MD5 signature of the views │
29520 │ │ for the design document │
29521 ├────────────────┼────────────────────────────┤
29522 │update_seq │ The update sequence of the │
29523 │ │ corresponding database │
29524 │ │ that has been indexed │
29525 ├────────────────┼────────────────────────────┤
29526 │updater_running │ Indicates if the view is │
29527 │ │ currently being updated │
29528 ├────────────────┼────────────────────────────┤
29529 │waiting_clients │ Number of clients waiting │
29530 │ │ on views from this design │
29531 │ │ document │
29532 ├────────────────┼────────────────────────────┤
29533 │waiting_commit │ Indicates if there are │
29534 │ │ outstanding commits to the │
29535 │ │ underlying database that │
29536 │ │ need to processed │
29537 └────────────────┴────────────────────────────┘
29538 Document with Attachments
29540 ┌────────────────────────┬────────────────────────────┐
29541 │Field │ Description │
29542 ├────────────────────────┼────────────────────────────┤
29543 │_id (optional) │ Document ID │
29544 ├────────────────────────┼────────────────────────────┤
29545 │_rev (optional) │ Revision ID (when updating │
29546 │ │ an existing document) │
29547 ├────────────────────────┼────────────────────────────┤
29548 │_attachments (optional) │ Document Attachment │
29549 ├────────────────────────┼────────────────────────────┤
29550 │filename │ Attachment information │
29551 ├────────────────────────┼────────────────────────────┤
29552 │content_type │ MIME Content type string │
29553 ├────────────────────────┼────────────────────────────┤
29554 │data │ File attachment content, │
29555 │ │ Base64 encoded │
29556 └────────────────────────┴────────────────────────────┘
29557 List of Active Tasks
29559 ┌──────────────┬─────────────────────┐
29560 │Field │ Description │
29561 ├──────────────┼─────────────────────┤
29562 │tasks [array] │ Active Tasks │
29563 ├──────────────┼─────────────────────┤
29564 │pid │ Process ID │
29565 ├──────────────┼─────────────────────┤
29566 │status │ Task status message │
29567 ├──────────────┼─────────────────────┤
29568 │task │ Task name │
29569 ├──────────────┼─────────────────────┤
29570 │type │ Operation Type │
29571 └──────────────┴─────────────────────┘
29572 Replication Settings
29574 ┌───────────────────────────┬────────────────────────────┐
29575 │Field │ Description │
29576 ├───────────────────────────┼────────────────────────────┤
29577 │source │ Source database name or │
29578 │ │ URL. │
29579 ├───────────────────────────┼────────────────────────────┤
29580 │target │ Target database name or │
29581 │ │ URL. │
29582 ├───────────────────────────┼────────────────────────────┤
29583 │cancel (optional) │ Cancels the replication. │
29584 ├───────────────────────────┼────────────────────────────┤
29585 │checkpoint_interval │ Specifies the checkpoint │
29586 │(optional) │ interval in ms. │
29587 ├───────────────────────────┼────────────────────────────┤
29588 │continuous (optional) │ Configure the replication │
29589 │ │ to be continuous. │
29590 ├───────────────────────────┼────────────────────────────┤
29591 │create_target (optional) │ Creates the target data‐ │
29592 │ │ base. │
29593 └───────────────────────────┴────────────────────────────┘
29594 │doc_ids (optional) │ Array of document IDs to │
29597 │ │ be synchronized. │
29598 ├───────────────────────────┼────────────────────────────┤
29599 │filter (optional) │ name of the filter func‐ │
29600 │ │ tion in the form of │
29601 │ │ ddoc/myfilter. │
29602 ├───────────────────────────┼────────────────────────────┤
29603 │source_proxy (optional) │ Address of a proxy server │
29604 │ │ through which replication │
29605 │ │ from the source should │
29606 │ │ occur. │
29607 ├───────────────────────────┼────────────────────────────┤
29608 │target_proxy (optional) │ Address of a proxy server │
29609 │ │ through which replication │
29610 │ │ to the target should │
29611 │ │ occur. │
29612 ├───────────────────────────┼────────────────────────────┤
29613 │query_params (optional) │ Query parameter that are │
29614 │ │ passed to the filter func‐ │
29615 │ │ tion; the value should be │
29616 │ │ a document containing │
29617 │ │ parameters as members. │
29618 ├───────────────────────────┼────────────────────────────┤
29619 │selector (optional) │ Select the documents │
29620 │ │ included in the replica‐ │
29621 │ │ tion. This option provides │
29622 │ │ performance benefits com‐ │
29623 │ │ pared with using the fil‐ │
29624 │ │ ter option. │
29625 ├───────────────────────────┼────────────────────────────┤
29626 │since_seq (optional) │ Sequence from which the │
29627 │ │ replication should start. │
29628 ├───────────────────────────┼────────────────────────────┤
29629 │use_checkpoints (optional) │ Whether to use replication │
29630 │ │ checkpoints or not. │
29631 └───────────────────────────┴────────────────────────────┘
29632 Replication Status
29634 ┌───────────────────┬────────────────────────────┐
29635 │Field │ Description │
29636 ├───────────────────┼────────────────────────────┤
29637 │ok │ Replication status │
29638 ├───────────────────┼────────────────────────────┤
29639 │session_id │ Unique session ID │
29640 ├───────────────────┼────────────────────────────┤
29641 │source_last_seq │ Last sequence number read │
29642 │ │ from the source database │
29643 ├───────────────────┼────────────────────────────┤
29644 │history [array] │ Replication History │
29645 ├───────────────────┼────────────────────────────┤
29646 │session_id │ Session ID for this repli‐ │
29647 │ │ cation operation │
29648 ├───────────────────┼────────────────────────────┤
29649 │recorded_seq │ Last recorded sequence │
29650 │ │ number │
29651 ├───────────────────┼────────────────────────────┤
29652 │docs_read │ Number of documents read │
29653 ├───────────────────┼────────────────────────────┤
29654 │docs_written │ Number of documents writ‐ │
29655 │ │ ten to target │
29656 ├───────────────────┼────────────────────────────┤
29657 │doc_write_failures │ Number of document write │
29658 │ │ failures │
29659 ├───────────────────┼────────────────────────────┤
29660 │start_time │ Date/Time replication │
29661 │ │ operation started │
29662 ├───────────────────┼────────────────────────────┤
29663 │start_last_seq │ First sequence number in │
29664 │ │ changes stream │
29665 ├───────────────────┼────────────────────────────┤
29666 │end_time │ Date/Time replication │
29667 │ │ operation completed │
29668 ├───────────────────┼────────────────────────────┤
29669 │end_last_seq │ Last sequence number in │
29670 │ │ changes stream │
29671 ├───────────────────┼────────────────────────────┤
29672 │missing_checked │ Number of missing docu‐ │
29673 │ │ ments checked │
29674 ├───────────────────┼────────────────────────────┤
29675 │missing_found │ Number of missing docu‐ │
29676 │ │ ments found │
29677 └───────────────────┴────────────────────────────┘
29678 Request object
29680 ┌───────────────┬────────────────────────────┐
29681 │Field │ Description │
29682 ├───────────────┼────────────────────────────┤
29683 │body │ Request body data as │
29684 │ │ string. If the request │
29685 │ │ method is GET this field │
29686 │ │ contains the value "unde‐ │
29687 │ │ fined". If the method is │
29688 │ │ DELETE or HEAD the value │
29689 │ │ is "" (empty string). │
29690 ├───────────────┼────────────────────────────┤
29691 │cookie │ Cookies object. │
29692 ├───────────────┼────────────────────────────┤
29693 │form │ Form data object. Con‐ │
29694 │ │ tains the decoded body as │
29695 │ │ key-value pairs if the │
29696 │ │ Content-Type header was │
29697 │ │ applica‐ │
29698 │ │ tion/x-www-form-urlen‐ │
29699 │ │ coded. │
29700 ├───────────────┼────────────────────────────┤
29701 │headers │ Request headers object. │
29702 ├───────────────┼────────────────────────────┤
29703 │id │ Requested document id │
29704 │ │ string if it was specified │
29705 │ │ or null otherwise. │
29706 ├───────────────┼────────────────────────────┤
29707 │info │ Database information │
29708 ├───────────────┼────────────────────────────┤
29709 │method │ Request method as string │
29710 │ │ or array. String value is │
29711 │ │ a method as one of: HEAD, │
29712 │ │ GET, POST, PUT, DELETE, │
29713 │ │ OPTIONS, and TRACE. Other‐ │
29714 │ │ wise it will be repre‐ │
29715 │ │ sented as an array of char │
29716 │ │ codes. │
29717 ├───────────────┼────────────────────────────┤
29718 │path │ List of requested path │
29719 │ │ sections. │
29720 ├───────────────┼────────────────────────────┤
29721 │peer │ Request source IP address. │
29722 ├───────────────┼────────────────────────────┤
29723 │query │ URL query parameters │
29724 │ │ object. Note that multi‐ │
29725 │ │ ple keys are not supported │
29726 │ │ and the last key value │
29727 │ │ suppresses others. │
29728 ├───────────────┼────────────────────────────┤
29729 │requested_path │ List of actual requested │
29730 │ │ path section. │
29731 └───────────────┴────────────────────────────┘
29732 │raw_path │ Raw requested path string. │
29734 ├───────────────┼────────────────────────────┤
29735 │secObj │ Security Object. │
29736 ├───────────────┼────────────────────────────┤
29737 │userCtx │ User Context Object. │
29738 ├───────────────┼────────────────────────────┤
29739 │uuid │ Generated UUID by a speci‐ │
29740 │ │ fied algorithm in the con‐ │
29741 │ │ fig file. │
29742 └───────────────┴────────────────────────────┘
29743 {
29745 "body": "undefined",
29746 "cookie": {
29747 "AuthSession": "cm9vdDo1MDZBRjQzRjrfcuikzPRfAn-EA37FmjyfM8G8Lw",
29748 "m": "3234"
29749 },
29750 "form": {},
29751 "headers": {
29752 "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
29753 "Accept-Charset": "ISO-8859-1,utf-8;q=0.7,*;q=0.3",
29754 "Accept-Encoding": "gzip,deflate,sdch",
29755 "Accept-Language": "en-US,en;q=0.8",
29756 "Connection": "keep-alive",
29757 "Cookie": "m=3234:t|3247:t|6493:t|6967:t|34e2:|18c3:t|2c69:t|5acb:t|ca3:t|c01:t|5e55:t|77cb:t|2a03:t|1d98:t|47ba:t|64b8:t|4a01:t; AuthSession=cm9vdDo1MDZBRjQzRjrfcuikzPRfAn-EA37FmjyfM8G8Lw",
29758 "Host": "127.0.0.1:5984",
29759 "User-Agent": "Mozilla/5.0 (Windows NT 5.2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7"
29760 },
29761 "id": "foo",
29762 "info": {
29763 "committed_update_seq": 2701412,
29764 "compact_running": false,
29765 "db_name": "mailbox",
29766 "disk_format_version": 6,
29767 "doc_count": 2262757,
29768 "doc_del_count": 560,
29769 "instance_start_time": "1347601025628957",
29770 "purge_seq": 0,
29771 "sizes": {
29772 "active": 7580843252,
29773 "disk": 14325313673,
29774 "external": 7803423459
29775 },
29776 "update_seq": 2701412
29777 },
29778 "method": "GET",
29779 "path": [
29780 "mailbox",
29781 "_design",
29782 "request",
29783 "_show",
29784 "dump",
29785 "foo"
29786 ],
29787 "peer": "127.0.0.1",
29788 "query": {},
29789 "raw_path": "/mailbox/_design/request/_show/dump/foo",
29790 "requested_path": [
29791 "mailbox",
29792 "_design",
29793 "request",
29794 "_show",
29795 "dump",
29796 "foo"
29797 ],
29798 "secObj": {
29799 "admins": {
29800 "names": [
29801 "Bob"
29802 ],
29803 "roles": []
29804 },
29805 "members": {
29806 "names": [
29807 "Mike",
29808 "Alice"
29809 ],
29810 "roles": []
29811 }
29812 },
29813 "userCtx": {
29814 "db": "mailbox",
29815 "name": "Mike",
29816 "roles": [
29817 "user"
29818 ]
29819 },
29820 "uuid": "3184f9d1ea934e1f81a24c71bde5c168"
29821 }
29822 Request2 object
29824 ┌───────────────┬────────────────────────────┐
29825 │Field │ Description │
29826 ├───────────────┼────────────────────────────┤
29827 │body │ Request body data as │
29828 │ │ string. If the request │
29829 │ │ method is GET this field │
29830 │ │ contains the value "unde‐ │
29831 │ │ fined". If the method is │
29832 │ │ DELETE or HEAD the value │
29833 │ │ is "" (empty string). │
29834 ├───────────────┼────────────────────────────┤
29835 │cookie │ Cookies object. │
29836 ├───────────────┼────────────────────────────┤
29837 │headers │ Request headers object. │
29838 ├───────────────┼────────────────────────────┤
29839 │method │ Request method as string │
29840 │ │ or array. String value is │
29841 │ │ a method as one of: HEAD, │
29842 │ │ GET, POST, PUT, DELETE, │
29843 │ │ OPTIONS, and TRACE. Other‐ │
29844 │ │ wise it will be repre‐ │
29845 │ │ sented as an array of char │
29846 │ │ codes. │
29847 ├───────────────┼────────────────────────────┤
29848 │path │ List of requested path │
29849 │ │ sections. │
29850 ├───────────────┼────────────────────────────┤
29851 │peer │ Request source IP address. │
29852 ├───────────────┼────────────────────────────┤
29853 │query │ URL query parameters │
29854 │ │ object. Note that multi‐ │
29855 │ │ ple keys are not supported │
29856 │ │ and the last key value │
29857 │ │ suppresses others. │
29858 ├───────────────┼────────────────────────────┤
29859 │requested_path │ List of actual requested │
29860 │ │ path section. │
29861 ├───────────────┼────────────────────────────┤
29862 │raw_path │ Raw requested path string. │
29863 ├───────────────┼────────────────────────────┤
29864 │secObj │ Security Object. │
29865 ├───────────────┼────────────────────────────┤
29866 │userCtx │ User Context Object. │
29867 └───────────────┴────────────────────────────┘
29868 Response object
29870 ───────────────────────────────────────
29871 Field Description
29872 ───────────────────────────────────────
29873 code HTTP status code number.
29874 ───────────────────────────────────────
29875 json JSON encodable object.
29876 Implicitly sets Con‐
29877 tent-Type header as appli‐
29878 cation/json.
29879 ───────────────────────────────────────
29880 body Raw response text string.
29881 Implicitly sets Con‐
29882 tent-Type header as
29883 text/html; charset=utf-8.
29884 ───────────────────────────────────────
29885 base64 Base64 encoded string.
29886 Implicitly sets Con‐
29887 tent-Type header as appli‐
29888 cation/binary.
29889 ───────────────────────────────────────
29890 headers Response headers object.
29891 Content-Type header from
29892 this object overrides any
29893 implicitly assigned one.
29894 ───────────────────────────────────────
29895 stop boolean signal to stop
29896 iteration over view result
29897 rows (for list functions
29898 only)
29899 ┌────────┬────────────────────────────┐
29900 │ │ │
29901 WARNING: │ │ │
29902 The body, bas│e64 and j│son object keys are overlap│ping each other
29903 where the l│ast one │wins. Since most realizat│ions of key-value
29904 objects do no│t preserv│e the key order or if they ar│e mixed, confus‐
29905 ing situation│s can occ│ur. Try to use only one of th│em.
29906 │ │ │
29907 NOTE: │ │ │
29908 Any custom p│roperty m│akes CouchDB raise an interna│l exception. Fur‐
29909 thermore, the│Response│object could be a simple str│ing value which
29910 would be impl│icitly wr│apped into a {"body": ...} ob│ject.
29911 │ │ │
29912 Returned CouchDB Doc│ument wit│h Detailed Revision Info │
29913 ┌────┼────────┼─────┬──────────────────────┼─────┐
29914 │Fiel│d │ │ Description │ │
29915 ├────┼────────┼─────┼──────────────────────┼─────┤
29916 │_id │(optional│) │ Document ID │ │
29917 ├────┼────────┼─────┼──────────────────────┼─────┤
29918 │_rev│(optiona│l) │ Revision ID (when upd│ating │
29919 │ │ │ │ an existing document)│ │
29920 ├────┼────────┼─────┼──────────────────────┼─────┤
29921 │_rev│s_info [a│rray] │ CouchDB document ext│ended │
29922 │ │ │ │ revision info │ │
29923 ├────┼────────┼─────┼──────────────────────┼─────┤
29924 │rev │ │ │ Full revision string │ │
29925 ├────┼────────┼─────┼──────────────────────┼─────┤
29926 │stat│us │ │ Status of the revisio│n │
29927 └────┼────────┼─────┴──────────────────────┼─────┘
29928 │ │ │
29929 Returned CouchDB Doc│ument wit│h Revision Info │
29930 ┌───┼────────┼───┬────────────────────────┼───┐
29931 │Fie│ld │ │ Description │ │
29932 ├───┼────────┼───┼────────────────────────┼───┤
29933 │_id│(optiona│l) │ Document ID │ │
29934 ├───┼────────┼───┼────────────────────────┼───┤
29935 │_re│v (option│al) │ Revision ID (when updat│ing │
29936 │ │ │ │ an existing document) │ │
29937 ├───┼────────┼───┼────────────────────────┼───┤
29938 │_re│visions │ │ CouchDB document revisi│ons │
29939 ├───┼────────┼───┼────────────────────────┼───┤
29940 │ids│[array] │ │ Array of valid revis│ion │
29941 │ │ │ │ IDs, in reverse or│der │
29942 │ │ │ │ (latest first) │ │
29943 ├───┼────────┼───┼────────────────────────┼───┤
29944 │sta│rt │ │ Prefix number for the l│at‐ │
29945 │ │ │ │ est revision │ │
29946 └───┼────────┼───┴────────────────────────┼───┘
29947 │ │ │
29948 Returned Document wi│th Attach│ments │
29949 ┌───────┼────────┼───────┬────────────────────┼───────┐
29950 │Field │ │ │ Description │ │
29951 ├───────┼────────┼───────┼────────────────────┼───────┤
29952 │_id (op│tional) │ │ Document ID │ │
29953 ├───────┼────────┼───────┼────────────────────┼───────┤
29954 │_rev (o│ptional) │ │ Revision ID (when u│pdating │
29955 │ │ │ │ an existing documen│t) │
29956 ├───────┼────────┼───────┼────────────────────┼───────┤
29957 │_attach│ments (op│tional) │ Document attachment│ │
29958 ├───────┼────────┼───────┼────────────────────┼───────┤
29959 │filenam│e │ │ Attachment │ │
29960 ├───────┼────────┼───────┼────────────────────┼───────┤
29961 │stub │ │ │ Indicates whether│ the │
29962 │ │ │ │ attachment is a stu│b │
29963 ├───────┼────────┼───────┼────────────────────┼───────┤
29964 │content│_type │ │ MIME Content type s│tring │
29965 ├───────┼────────┼───────┼────────────────────┼───────┤
29966 │length │ │ │ Length (bytes) │of the │
29967 │ │ │ │ attachment data │ │
29968 ├───────┼────────┼───────┼────────────────────┼───────┤
29969 │revpos │ │ │ Revision where │ this │
29970 │ │ │ │ attachment exists │ │
29971 └───────┼────────┼───────┴────────────────────┼───────┘
29972 │ │ │
29973 Security Object │ │ │
29974 ┌──┼────────┼──┬─────────────────────────┼──┐
29975 │Fi│eld │ │ Description │ │
29976 ├──┼────────┼──┼─────────────────────────┼──┤
29977 │ad│mins │ │ Roles/Users with adm│in │
29978 │ │ │ │ privileges │ │
29979 ├──┼────────┼──┼─────────────────────────┼──┤
29980 │ro│les [arra│y] │ List of roles with pare│nt │
29981 │ │ │ │ privilege │ │
29982 ├──┼────────┼──┼─────────────────────────┼──┤
29983 │na│mes [arra│y] │ List of users with pare│nt │
29984 │ │ │ │ privilege │ │
29985 ├──┼────────┼──┼─────────────────────────┼──┤
29986 │me│mbers │ │ Roles/Users with non-adm│in │
29987 │ │ │ │ privileges │ │
29988 ├──┼────────┼──┼─────────────────────────┼──┤
29989 │ro│les [arra│y] │ List of roles with pare│nt │
29990 │ │ │ │ privilege │ │
29991 ├──┼────────┼──┼─────────────────────────┼──┤
29992 │na│mes [arra│y] │ List of users with pare│nt │
29993 │ │ │ │ privilege │ │
29994 └──┼────────┼──┴─────────────────────────┼──┘
29995 │ │ │
29996 { │ │ │
29997 "admins":│{ │ │
29998 "name│s": [ │ │
29999 "│Bob" │ │
30000 ], │ │ │
30001 "role│s": [] │ │
30002 }, │ │ │
30003 "members"│: { │ │
30004 "name│s": [ │ │
30005 "│Mike", │ │
30006 "│Alice" │ │
30007 ], │ │ │
30008 "role│s": [] │ │
30009 } │ │ │
30010 }
30011 User Context Object
30013 ┌──────┬────────────────────────────┐
30014 │Field │ Description │
30015 ├──────┼────────────────────────────┤
30016 │db │ Database name in the con‐ │
30017 │ │ text of the provided oper‐ │
30018 │ │ ation. │
30019 ├──────┼────────────────────────────┤
30020 │name │ User name. │
30021 ├──────┼────────────────────────────┤
30022 │roles │ List of user roles. │
30023 └──────┴────────────────────────────┘
30024 {
30026 "db": "mailbox",
30027 "name": null,
30028 "roles": [
30029 "_admin"
30030 ]
30031 }
30032 View Head Information
30034 ┌───────────┬────────────────────────────┐
30035 │Field │ Description │
30036 ├───────────┼────────────────────────────┤
30037 │total_rows │ Number of documents in the │
30038 │ │ view │
30039 ├───────────┼────────────────────────────┤
30040 │offset │ Offset where the document │
30041 │ │ list started │
30042 └───────────┴────────────────────────────┘
30043 {
30045 "total_rows": 42,
30046 "offset": 3
30047 }
30048
30050 The Query server is an external process that communicates with CouchDB
30051 by JSON protocol through stdio interface and processes all design func‐
30052 tions calls, such as JavaScript views.
30053 The default query server is written in JavaScript, running via Mozilla
30055 SpiderMonkey. You can use other languages by setting a Query server
30056 key in the language property of a design document or the Content-Type
30057 header of a temporary view. Design documents that do not specify a lan‐
30058 guage property are assumed to be of type javascript.
30059 Query Server Protocol
30061 A Query Server is an external process that communicates with CouchDB
30062 via a simple, custom JSON protocol over stdin/stdout. It is used to
30063 processes all design functions calls: views, shows, lists, filters,
30064 updates and validate_doc_update.
30065 CouchDB communicates with the Query Server process through stdin/stdout
30067 with JSON messages that are terminated by a newline character. Messages
30068 that are sent to the Query Server are always array-typed and follow the
30069 pattern [<command>, <*arguments>]\n.
30070 NOTE:
30072 In the documentation examples, we omit the trailing \n for greater
30073 readability. Also, examples contain formatted JSON values while real
30074 data is transferred in compact mode without formatting spaces.
30075 reset
30077 Command
30078 reset
30079 Arguments
30081 Query server state (optional)
30082 Returns
30084 true
30085 This resets the state of the Query Server and makes it forget all pre‐
30087 vious input. If applicable, this is the point to run garbage collec‐
30088 tion.
30089 CouchDB sends:
30091 ["reset"]
30093 The Query Server answers:
30095 true
30097 To set up new Query Server state, the second argument is used with
30099 object data.
30100 CouchDB sends:
30102 ["reset", {"reduce_limit": true, "timeout": 5000}]
30104 The Query Server answers:
30106 true
30108 add_lib
30110 Command
30111 add_lib
30112 Arguments
30114 CommonJS library object by views/lib path
30115 Returns
30117 true
30118 Adds CommonJS library to Query Server state for further usage in map
30120 functions.
30121 CouchDB sends:
30123 [
30125 "add_lib",
30126 {
30127 "utils": "exports.MAGIC = 42;"
30128 }
30129 ]
30130 The Query Server answers:
30132 true
30134 NOTE:
30136 This library shouldn’t have any side effects nor track its own state
30137 or you’ll have a lot of happy debugging time if something goes
30138 wrong. Remember that a complete index rebuild is a heavy operation
30139 and this is the only way to fix mistakes with shared state.
30140 add_fun
30142 Command
30143 add_fun
30144 Arguments
30146 Map function source code.
30147 Returns
30149 true
30150 When creating or updating a view, this is how the Query Server is sent
30152 the view function for evaluation. The Query Server should parse, com‐
30153 pile, and evaluate the function it receives to make it callable later.
30154 If this fails, the Query Server returns an error. CouchDB may store
30155 multiple functions before sending any documents.
30156 CouchDB sends:
30158 [
30160 "add_fun",
30161 "function(doc) { if(doc.score > 50) emit(null, {'player_name': doc.name}); }"
30162 ]
30163 The Query Server answers:
30165 true
30167 map_doc
30169 Command
30170 map_doc
30171 Arguments
30173 Document object
30174 Returns
30176 Array of key-value pairs per applied function
30177 When the view function is stored in the Query Server, CouchDB starts
30179 sending all the documents in the database, one at a time. The Query
30180 Server calls the previously stored functions one after another with a
30181 document and stores its result. When all functions have been called,
30182 the result is returned as a JSON string.
30183 CouchDB sends:
30185 [
30187 "map_doc",
30188 {
30189 "_id": "8877AFF9789988EE",
30190 "_rev": "3-235256484",
30191 "name": "John Smith",
30192 "score": 60
30193 }
30194 ]
30195 If the function above is the only function stored, the Query Server
30197 answers:
30198 [
30200 [
30201 [null, {"player_name": "John Smith"}]
30202 ]
30203 ]
30204 That is, an array with the result for every function for the given doc‐
30206 ument.
30207 If a document is to be excluded from the view, the array should be
30209 empty.
30210 CouchDB sends:
30212 [
30214 "map_doc",
30215 {
30216 "_id": "9590AEB4585637FE",
30217 "_rev": "1-674684684",
30218 "name": "Jane Parker",
30219 "score": 43
30220 }
30221 ]
30222 The Query Server answers:
30224 [[]]
30226 reduce
30228 Command
30229 reduce
30230 Arguments
30232 · Reduce function source
30234 · Array of map function results where each item represented in
30236 format [[key, id-of-doc], value]
30237 Returns
30239 Array with pair values: true and another array with reduced
30240 result
30241 If the view has a reduce function defined, CouchDB will enter into the
30243 reduce phase. The Query Server will receive a list of reduce functions
30244 and some map results on which it can apply them.
30245 CouchDB sends:
30247 [
30249 "reduce",
30250 [
30251 "function(k, v) { return sum(v); }"
30252 ],
30253 [
30254 [[1, "699b524273605d5d3e9d4fd0ff2cb272"], 10],
30255 [[2, "c081d0f69c13d2ce2050d684c7ba2843"], 20],
30256 [[null, "foobar"], 3]
30257 ]
30258 ]
30259 The Query Server answers:
30261 [
30263 true,
30264 [33]
30265 ]
30266 Note that even though the view server receives the map results in the
30268 form [[key, id-of-doc], value], the function may receive them in a dif‐
30269 ferent form. For example, the JavaScript Query Server applies functions
30270 on the list of keys and the list of values.
30271 rereduce
30273 Command
30274 rereduce
30275 Arguments
30277 · Reduce function source
30279 · List of values
30281 When building a view, CouchDB will apply the reduce step directly to
30283 the output of the map step and the rereduce step to the output of a
30284 previous reduce step.
30285 CouchDB will send a list of reduce functions and a list of values, with
30287 no keys or document ids to the rereduce step.
30288 CouchDB sends:
30290 [
30292 "rereduce",
30293 [
30294 "function(k, v, r) { return sum(v); }"
30295 ],
30296 [
30297 33,
30298 55,
30299 66
30300 ]
30301 ]
30302 The Query Server answers:
30304 [
30306 true,
30307 [154]
30308 ]
30309 ddoc
30311 Command
30312 ddoc
30313 Arguments
30315 Array of objects.
30316 · First phase (ddoc initialization):
30318 · "new"
30320 · Design document _id
30322 · Design document object
30324 · Second phase (design function execution):
30326 · Design document _id
30328 · Function path as an array of object keys
30330 · Array of function arguments
30332 Returns
30334 · First phase (ddoc initialization): true
30336 · Second phase (design function execution): custom object
30338 depending on executed function
30339 This command acts in two phases: ddoc registration and design function
30341 execution.
30342 In the first phase CouchDB sends a full design document content to the
30344 Query Server to let it cache it by _id value for further function exe‐
30345 cution.
30346 To do this, CouchDB sends:
30348 [
30350 "ddoc",
30351 "new",
30352 "_design/temp",
30353 {
30354 "_id": "_design/temp",
30355 "_rev": "8-d7379de23a751dc2a19e5638a7bbc5cc",
30356 "language": "javascript",
30357 "shows": {
30358 "request": "function(doc,req){ return {json: req}; }",
30359 "hello": "function(doc,req){ return {body: 'Hello, ' + (doc || {})._id + '!'}; }"
30360 }
30361 }
30362 ]
30363 The Query Server answers:
30365 true
30367 After this, the design document will be ready to serve subcommands in
30369 the second phase.
30370 NOTE:
30372 Each ddoc subcommand is the root design document key, so they are
30373 not actually subcommands, but first elements of the JSON path that
30374 may be handled and processed.
30375 The pattern for subcommand execution is common:
30377 ["ddoc", <design_doc_id>, [<subcommand>, <funcname>], [<argument1>,
30379 <argument2>, ...]]
30380 shows
30382 WARNING:
30383 Show functions are deprecated in CouchDB 3.0, and will be removed in
30384 CouchDB 4.0.
30385 Command
30387 ddoc
30388 SubCommand
30390 shows
30391 Arguments
30393 · Document object or null if document id isn’t specified in
30395 request
30396 · request_object
30398 Returns
30400 Array with two elements:
30401 · "resp"
30403 · response_object
30405 Executes show function.
30407 Couchdb sends:
30409 [
30411 "ddoc",
30412 "_design/temp",
30413 [
30414 "shows",
30415 "doc"
30416 ],
30417 [
30418 null,
30419 {
30420 "info": {
30421 "db_name": "test",
30422 "doc_count": 8,
30423 "doc_del_count": 0,
30424 "update_seq": 105,
30425 "purge_seq": 0,
30426 "compact_running": false,
30427 "sizes": {
30428 "active": 1535048,
30429 "disk": 15818856,
30430 "external": 15515850
30431 },
30432 "instance_start_time": "1359952188595857",
30433 "disk_format_version": 6,
30434 "committed_update_seq": 105
30435 },
30436 "id": null,
30437 "uuid": "169cb4cc82427cc7322cb4463d0021bb",
30438 "method": "GET",
30439 "requested_path": [
30440 "api",
30441 "_design",
30442 "temp",
30443 "_show",
30444 "request"
30445 ],
30446 "path": [
30447 "api",
30448 "_design",
30449 "temp",
30450 "_show",
30451 "request"
30452 ],
30453 "raw_path": "/api/_design/temp/_show/request",
30454 "query": {},
30455 "headers": {
30456 "Accept": "*/*",
30457 "Host": "localhost:5984",
30458 "User-Agent": "curl/7.26.0"
30459 },
30460 "body": "undefined",
30461 "peer": "127.0.0.1",
30462 "form": {},
30463 "cookie": {},
30464 "userCtx": {
30465 "db": "api",
30466 "name": null,
30467 "roles": [
30468 "_admin"
30469 ]
30470 },
30471 "secObj": {}
30472 }
30473 ]
30474 ]
30475 The Query Server sends:
30477 [
30479 "resp",
30480 {
30481 "body": "Hello, undefined!"
30482 }
30483 ]
30484 lists
30486 WARNING:
30487 List functions are deprecated in CouchDB 3.0, and will be removed in
30488 CouchDB 4.0.
30489 Command
30491 ddoc
30492 SubCommand
30494 lists
30495 Arguments
30497 · view_head_info_object:
30499 · request_object
30501 Returns
30503 Array. See below for details.
30504 Executes list function.
30506 The communication protocol for list functions is a bit complex so let’s
30508 use an example to illustrate.
30509 Assume we have view a function that emits id-rev pairs:
30511 function(doc) {
30513 emit(doc._id, doc._rev);
30514 }
30515 And we’d like to emulate _all_docs JSON response with list function.
30517 Our first version of the list functions looks like this:
30518 function(head, req){
30520 start({'headers': {'Content-Type': 'application/json'}});
30521 var resp = head;
30522 var rows = [];
30523 while(row=getRow()){
30524 rows.push(row);
30525 }
30526 resp.rows = rows;
30527 return toJSON(resp);
30528 }
30529 The whole communication session during list function execution could be
30531 divided on three parts:
30532 1. Initialization
30534 The first returned object from the list function is an array with
30536 the following structure:
30537 ["start", <chunks>, <headers>]
30539 Where <chunks> is an array of text chunks that will be sent to the
30541 client and <headers> is an object with response HTTP headers.
30542 This message is sent from the Query Server to CouchDB on the start()
30544 call which initializes the HTTP response to the client:
30545 [
30547 "start",
30548 [],
30549 {
30550 "headers": {
30551 "Content-Type": "application/json"
30552 }
30553 }
30554 ]
30555 After this, the list function may start to process view rows.
30557 2. View Processing
30559 Since view results can be extremely large, it is not wise to pass
30561 all its rows in a single command. Instead, CouchDB can send view
30562 rows one by one to the Query Server allowing view processing and
30563 output generation to be processed as a stream.
30564 CouchDB sends a special array that carries view row data:
30566 [
30568 "list_row",
30569 {
30570 "id": "0cb42c267fe32d4b56b3500bc503e030",
30571 "key": "0cb42c267fe32d4b56b3500bc503e030",
30572 "value": "1-967a00dff5e02add41819138abb3284d"
30573 }
30574 ]
30575 If the Query Server has something to return on this, it returns an
30577 array with a "chunks" item in the head and an array of data in the
30578 tail. For this example it has nothing to return, so the response
30579 will be:
30580 [
30582 "chunks",
30583 []
30584 ]
30585 When there are no more view rows to process, CouchDB sends a
30587 list_end message to signify there is no more data to send:
30588 ["list_end"]
30590 3. Finalization
30592 The last stage of the communication process is the returning list
30594 tail: the last data chunk. After this, processing of the list func‐
30595 tion will be complete and the client will receive a complete
30596 response.
30597 For our example the last message is:
30599 [
30601 "end",
30602 [
30603 "{\"total_rows\":2,\"offset\":0,\"rows\":[{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"},{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}]}"
30604 ]
30605 ]
30606 In this example, we have returned our result in a single message from
30608 the Query Server. This is okay for small numbers of rows, but for large
30609 data sets, perhaps with millions of documents or millions of view rows,
30610 this would not be acceptable.
30611 Let’s fix our list function and see the changes in communication:
30613 function(head, req){
30615 start({'headers': {'Content-Type': 'application/json'}});
30616 send('{');
30617 send('"total_rows":' + toJSON(head.total_rows) + ',');
30618 send('"offset":' + toJSON(head.offset) + ',');
30619 send('"rows":[');
30620 if (row=getRow()){
30621 send(toJSON(row));
30622 }
30623 while(row=getRow()){
30624 send(',' + toJSON(row));
30625 }
30626 send(']');
30627 return '}';
30628 }
30629 “Wait, what?” - you’d like to ask. Yes, we’d build JSON response manu‐
30631 ally by string chunks, but let’s take a look on logs:
30632 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["start",["{","\"total_rows\":2,","\"offset\":0,","\"rows\":["],{"headers":{"Content-Type":"application/json"}}]
30634 [Wed, 24 Jul 2013 05:45:30 GMT] [info] [<0.18963.1>] 127.0.0.1 - - GET /blog/_design/post/_list/index/all_docs 200
30635 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"0cb42c267fe32d4b56b3500bc503e030","key":"0cb42c267fe32d4b56b3500bc503e030","value":"1-967a00dff5e02add41819138abb3284d"}]
30636 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",["{\"id\":\"0cb42c267fe32d4b56b3500bc503e030\",\"key\":\"0cb42c267fe32d4b56b3500bc503e030\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
30637 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_row",{"id":"431926a69504bde41851eb3c18a27b1f","key":"431926a69504bde41851eb3c18a27b1f","value":"1-967a00dff5e02add41819138abb3284d"}]
30638 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["chunks",[",{\"id\":\"431926a69504bde41851eb3c18a27b1f\",\"key\":\"431926a69504bde41851eb3c18a27b1f\",\"value\":\"1-967a00dff5e02add41819138abb3284d\"}"]]
30639 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Input :: ["list_end"]
30640 [Wed, 24 Jul 2013 05:45:30 GMT] [debug] [<0.19191.1>] OS Process #Port<0.4444> Output :: ["end",["]","}"]]
30641 Note, that now the Query Server sends response by lightweight chunks
30643 and if our communication process was extremely slow, the client will
30644 see how response data appears on their screen. Chunk by chunk, without
30645 waiting for the complete result, like they have for our previous list
30646 function.
30647 updates
30649 Command
30650 ddoc
30651 SubCommand
30653 updates
30654 Arguments
30656 · Document object or null if document id wasn’t specified in
30658 request
30659 · request_object
30661 Returns
30663 Array with there elements:
30664 · "up"
30666 · Document object or null if nothing should be stored
30668 · response_object
30670 Executes update function.
30672 CouchDB sends:
30674 [
30676 "ddoc",
30677 "_design/id",
30678 [
30679 "updates",
30680 "nothing"
30681 ],
30682 [
30683 null,
30684 {
30685 "info": {
30686 "db_name": "test",
30687 "doc_count": 5,
30688 "doc_del_count": 0,
30689 "update_seq": 16,
30690 "purge_seq": 0,
30691 "compact_running": false,
30692 "sizes": {
30693 "active": 7979745,
30694 "disk": 8056936,
30695 "external": 8024930
30696 },
30697 "instance_start_time": "1374612186131612",
30698 "disk_format_version": 6,
30699 "committed_update_seq": 16
30700 },
30701 "id": null,
30702 "uuid": "7b695cb34a03df0316c15ab529002e69",
30703 "method": "POST",
30704 "requested_path": [
30705 "test",
30706 "_design",
30707 "1139",
30708 "_update",
30709 "nothing"
30710 ],
30711 "path": [
30712 "test",
30713 "_design",
30714 "1139",
30715 "_update",
30716 "nothing"
30717 ],
30718 "raw_path": "/test/_design/1139/_update/nothing",
30719 "query": {},
30720 "headers": {
30721 "Accept": "*/*",
30722 "Accept-Encoding": "identity, gzip, deflate, compress",
30723 "Content-Length": "0",
30724 "Host": "localhost:5984"
30725 },
30726 "body": "",
30727 "peer": "127.0.0.1",
30728 "form": {},
30729 "cookie": {},
30730 "userCtx": {
30731 "db": "test",
30732 "name": null,
30733 "roles": [
30734 "_admin"
30735 ]
30736 },
30737 "secObj": {}
30738 }
30739 ]
30740 ]
30741 The Query Server answers:
30743 [
30745 "up",
30746 null,
30747 {"body": "document id wasn't provided"}
30748 ]
30749 or in case of successful update:
30751 [
30753 "up",
30754 {
30755 "_id": "7b695cb34a03df0316c15ab529002e69",
30756 "hello": "world!"
30757 },
30758 {"body": "document was updated"}
30759 ]
30760 filters
30762 Command
30763 ddoc
30764 SubCommand
30766 filters
30767 Arguments
30769 · Array of document objects
30771 · request_object
30773 Returns
30775 Array of two elements:
30776 · true
30778 · Array of booleans in the same order of input documents.
30780 Executes filter function.
30782 CouchDB sends:
30784 [
30786 "ddoc",
30787 "_design/test",
30788 [
30789 "filters",
30790 "random"
30791 ],
30792 [
30793 [
30794 {
30795 "_id": "431926a69504bde41851eb3c18a27b1f",
30796 "_rev": "1-967a00dff5e02add41819138abb3284d",
30797 "_revisions": {
30798 "start": 1,
30799 "ids": [
30800 "967a00dff5e02add41819138abb3284d"
30801 ]
30802 }
30803 },
30804 {
30805 "_id": "0cb42c267fe32d4b56b3500bc503e030",
30806 "_rev": "1-967a00dff5e02add41819138abb3284d",
30807 "_revisions": {
30808 "start": 1,
30809 "ids": [
30810 "967a00dff5e02add41819138abb3284d"
30811 ]
30812 }
30813 }
30814 ],
30815 {
30816 "info": {
30817 "db_name": "test",
30818 "doc_count": 5,
30819 "doc_del_count": 0,
30820 "update_seq": 19,
30821 "purge_seq": 0,
30822 "compact_running": false,
30823 "sizes": {
30824 "active": 7979745,
30825 "disk": 8056936,
30826 "external": 8024930
30827 },
30828 "instance_start_time": "1374612186131612",
30829 "disk_format_version": 6,
30830 "committed_update_seq": 19
30831 },
30832 "id": null,
30833 "uuid": "7b695cb34a03df0316c15ab529023a81",
30834 "method": "GET",
30835 "requested_path": [
30836 "test",
30837 "_changes?filter=test",
30838 "random"
30839 ],
30840 "path": [
30841 "test",
30842 "_changes"
30843 ],
30844 "raw_path": "/test/_changes?filter=test/random",
30845 "query": {
30846 "filter": "test/random"
30847 },
30848 "headers": {
30849 "Accept": "application/json",
30850 "Accept-Encoding": "identity, gzip, deflate, compress",
30851 "Content-Length": "0",
30852 "Content-Type": "application/json; charset=utf-8",
30853 "Host": "localhost:5984"
30854 },
30855 "body": "",
30856 "peer": "127.0.0.1",
30857 "form": {},
30858 "cookie": {},
30859 "userCtx": {
30860 "db": "test",
30861 "name": null,
30862 "roles": [
30863 "_admin"
30864 ]
30865 },
30866 "secObj": {}
30867 }
30868 ]
30869 ]
30870 The Query Server answers:
30872 [
30874 true,
30875 [
30876 true,
30877 false
30878 ]
30879 ]
30880 views
30882 Command
30883 ddoc
30884 SubCommand
30886 views
30887 Arguments
30889 Array of document objects
30890 Returns
30892 Array of two elements:
30893 · true
30895 · Array of booleans in the same order of input documents.
30897 New in version 1.2.
30899 Executes view function in place of the filter.
30902 Acts in the same way as filters command.
30904 validate_doc_update
30906 Command
30907 ddoc
30908 SubCommand
30910 validate_doc_update
30911 Arguments
30913 · Document object that will be stored
30915 · Document object that will be replaced
30917 · userctx_object
30919 · security_object
30921 Returns
30923 1
30924 Executes validation function.
30926 CouchDB send:
30928 [
30930 "ddoc",
30931 "_design/id",
30932 ["validate_doc_update"],
30933 [
30934 {
30935 "_id": "docid",
30936 "_rev": "2-e0165f450f6c89dc6b071c075dde3c4d",
30937 "score": 10
30938 },
30939 {
30940 "_id": "docid",
30941 "_rev": "1-9f798c6ad72a406afdbf470b9eea8375",
30942 "score": 4
30943 },
30944 {
30945 "name": "Mike",
30946 "roles": ["player"]
30947 },
30948 {
30949 "admins": {},
30950 "members": []
30951 }
30952 ]
30953 ]
30954 The Query Server answers:
30956 1
30958 NOTE:
30960 While the only valid response for this command is true, to prevent
30961 the document from being saved, the Query Server needs to raise an
30962 error: forbidden or unauthorized; these errors will be turned into
30963 correct HTTP 403 and HTTP 401 responses respectively.
30964 rewrites
30966 Command
30967 ddoc
30968 SubCommand
30970 rewrites
30971 Arguments
30973 · request2_object
30975 Returns
30977 1
30978 Executes rewrite function.
30980 CouchDB send:
30982 [
30984 "ddoc",
30985 "_design/id",
30986 ["rewrites"],
30987 [
30988 {
30989 "method": "POST",
30990 "requested_path": [
30991 "test",
30992 "_design",
30993 "1139",
30994 "_update",
30995 "nothing"
30996 ],
30997 "path": [
30998 "test",
30999 "_design",
31000 "1139",
31001 "_update",
31002 "nothing"
31003 ],
31004 "raw_path": "/test/_design/1139/_update/nothing",
31005 "query": {},
31006 "headers": {
31007 "Accept": "*/*",
31008 "Accept-Encoding": "identity, gzip, deflate, compress",
31009 "Content-Length": "0",
31010 "Host": "localhost:5984"
31011 },
31012 "body": "",
31013 "peer": "127.0.0.1",
31014 "cookie": {},
31015 "userCtx": {
31016 "db": "test",
31017 "name": null,
31018 "roles": [
31019 "_admin"
31020 ]
31021 },
31022 "secObj": {}
31023 }
31024 ]
31025 ]
31026 The Query Server answers:
31028 [
31030 "ok",
31031 {
31032 "path": "some/path",
31033 "query": {"key1": "value1", "key2": "value2"},
31034 "method": "METHOD",
31035 "headers": {"Header1": "value1", "Header2": "value2"},
31036 "body": ""
31037 }
31038 ]
31039 or in case of direct response:
31041 [
31043 "ok",
31044 {
31045 "headers": {"Content-Type": "text/plain"},
31046 "body": "Welcome!",
31047 "code": 200
31048 }
31049 ]
31050 or for immediate redirect:
31052 [
31054 "ok",
31055 {
31056 "headers": {"Location": "http://example.com/path/"},
31057 "code": 302
31058 }
31059 ]
31060 Returning errors
31062 When something goes wrong, the Query Server can inform CouchDB by send‐
31063 ing a special message in response to the received command.
31064 Error messages prevent further command execution and return an error
31066 description to CouchDB. Errors are logically divided into two groups:
31067 · Common errors. These errors only break the current Query Server com‐
31069 mand and return the error info to the CouchDB instance without termi‐
31070 nating the Query Server process.
31071 · Fatal errors. Fatal errors signal a condition that cannot be recov‐
31073 ered. For instance, if your a design function is unable to import a
31074 third party module, it’s better to count such error as fatal and ter‐
31075 minate whole process.
31076 error
31078 To raise an error, the Query Server should respond with:
31079 ["error", "error_name", "reason why"]
31081 The "error_name" helps to classify problems by their type e.g. if it’s
31083 "value_error" to indicate improper data, "not_found" to indicate a
31084 missing resource and "type_error" to indicate an improper data type.
31085 The "reason why" explains in human-readable terms what went wrong, and
31087 possibly how to resolve it.
31088 For example, calling updatefun against a non-existent document could
31090 produce the error message:
31091 ["error", "not_found", "Update function requires existent document"]
31093 forbidden
31095 The forbidden error is widely used by vdufun to stop further function
31096 processing and prevent storage of the new document revision. Since this
31097 is not actually an error, but an assertion against user actions,
31098 CouchDB doesn’t log it at “error” level, but returns HTTP 403 Forbidden
31099 response with error information object.
31100 To raise this error, the Query Server should respond with:
31102 {"forbidden": "reason why"}
31104 unauthorized
31106 The unauthorized error mostly acts like forbidden one, but with the
31107 meaning of please authorize first. This small difference helps end
31108 users to understand what they can do to solve the problem. Similar to
31109 forbidden, CouchDB doesn’t log it at “error” level, but returns a HTTP
31110 401 Unauthorized response with an error information object.
31111 To raise this error, the Query Server should respond with:
31113 {"unauthorized": "reason why"}
31115 Logging
31117 At any time, the Query Server may send some information that will be
31118 saved in CouchDB’s log file. This is done by sending a special log
31119 object with a single argument, on a separate line:
31120 ["log", "some message"]
31122 CouchDB does not respond, but writes the received message to the log
31124 file:
31125 [Sun, 13 Feb 2009 23:31:30 GMT] [info] [<0.72.0>] Query Server Log Message: some message
31127 These messages are only logged at info level.
31129 JavaScript
31131 NOTE:
31132 While every design function has access to all JavaScript objects,
31133 the table below describes appropriate usage cases. For example, you
31134 may use emit() in mapfun, but getRow() is not permitted during map‐
31135 fun.
31136 ┌───────────────┬────────────────────────────┐
31138 │JS Function │ Reasonable to use in │
31139 │ │ design doc functions │
31140 ├───────────────┼────────────────────────────┤
31141 │emit() │ mapfun │
31142 ├───────────────┼────────────────────────────┤
31143 │getRow() │ listfun │
31144 ├───────────────┼────────────────────────────┤
31145 │JSON │ any │
31146 ├───────────────┼────────────────────────────┤
31147 │isArray() │ any │
31148 ├───────────────┼────────────────────────────┤
31149 │log() │ any │
31150 ├───────────────┼────────────────────────────┤
31151 │provides() │ showfun, listfun │
31152 ├───────────────┼────────────────────────────┤
31153 │registerType() │ showfun, listfun │
31154 ├───────────────┼────────────────────────────┤
31155 │require() │ any, except reducefun │
31156 ├───────────────┼────────────────────────────┤
31157 │send() │ listfun │
31158 ├───────────────┼────────────────────────────┤
31159 │start() │ listfun │
31160 ├───────────────┼────────────────────────────┤
31161 │sum() │ any │
31162 ├───────────────┼────────────────────────────┤
31163 │toJSON() │ any │
31164 └───────────────┴────────────────────────────┘
31165 Design functions context
31167 Each design function executes in a special context of predefined
31168 objects, modules and functions:
31169 emit(key, value)
31171 Emits a key-value pair for further processing by CouchDB after
31172 the map function is done.
31173 Arguments
31175 · key – The view key
31177 · value – The key’s associated value
31179 function(doc){
31181 emit(doc._id, doc._rev);
31182 }
31183 getRow()
31185 Extracts the next row from a related view result.
31186 Returns
31188 View result row
31189 Return type
31191 object
31192 function(head, req){
31194 send('[');
31195 row = getRow();
31196 if (row){
31197 send(toJSON(row));
31198 while(row = getRow()){
31199 send(',');
31200 send(toJSON(row));
31201 }
31202 }
31203 return ']';
31204 }
31205 JSON JSON2 object.
31207 isArray(obj)
31209 A helper function to check if the provided value is an Array.
31210 Arguments
31212 · obj – Any JavaScript value
31214 Returns
31216 true if obj is Array-typed, false otherwise
31217 Return type
31219 boolean
31220 log(message)
31222 Log a message to the CouchDB log (at the INFO level).
31223 Arguments
31225 · message – Message to be logged
31227 function(doc){
31229 log('Procesing doc ' + doc['_id']);
31230 emit(doc['_id'], null);
31231 }
31232 After the map function has run, the following line can be found
31234 in CouchDB logs (e.g. at /var/log/couchdb/couch.log):
31235 [Sat, 03 Nov 2012 17:38:02 GMT] [info] [<0.7543.0>] OS Process #Port<0.3289> Log :: Processing doc 8d300b86622d67953d102165dbe99467
31237 provides(key, func)
31239 Registers callable handler for specified MIME key.
31240 Arguments
31242 · key – MIME key previously defined by registerType()
31244 · func – MIME type handler
31246 registerType(key, *mimes)
31248 Registers list of MIME types by associated key.
31249 Arguments
31251 · key – MIME types
31253 · mimes – MIME types enumeration
31255 Predefined mappings (key-array):
31257 · all: */*
31259 · text: text/plain; charset=utf-8, txt
31261 · html: text/html; charset=utf-8
31263 · xhtml: application/xhtml+xml, xhtml
31265 · xml: application/xml, text/xml, application/x-xml
31267 · js: text/javascript, application/javascript, applica‐
31269 tion/x-javascript
31270 · css: text/css
31272 · ics: text/calendar
31274 · csv: text/csv
31276 · rss: application/rss+xml
31278 · atom: application/atom+xml
31280 · yaml: application/x-yaml, text/yaml
31282 · multipart_form: multipart/form-data
31284 · url_encoded_form: application/x-www-form-urlencoded
31286 · json: application/json, text/x-json
31288 require(path)
31290 Loads CommonJS module by a specified path. The path should not
31291 start with a slash.
31292 Arguments
31294 · path – A CommonJS module path started from design docu‐
31296 ment root
31297 Returns
31299 Exported statements
31300 send(chunk)
31302 Sends a single string chunk in response.
31303 Arguments
31305 · chunk – Text chunk
31307 function(head, req){
31309 send('Hello,');
31310 send(' ');
31311 send('Couch');
31312 return ;
31313 }
31314 start(init_resp)
31316 Initiates chunked response. As an option, a custom response
31317 object may be sent at this point. For list-functions only!
31318 NOTE:
31320 list functions may set the HTTP response code and headers by
31321 calling this function. This function must be called before
31322 send(), getRow() or a return statement; otherwise, the query
31323 server will implicitly call this function with the empty
31324 object ({}).
31325 function(head, req){
31327 start({
31328 "code": 302,
31329 "headers": {
31330 "Location": "http://couchdb.apache.org"
31331 }
31332 });
31333 return "Relax!";
31334 }
31335 sum(arr)
31337 Sum arr’s items.
31338 Arguments
31340 · arr – Array of numbers
31342 Return type
31344 number
31345 toJSON(obj)
31347 Encodes obj to JSON string. This is an alias for the
31348 JSON.stringify method.
31349 Arguments
31351 · obj – JSON-encodable object
31353 Returns
31355 JSON string
31356 CommonJS Modules
31358 Support for CommonJS Modules (introduced in CouchDB 0.11.0) allows you
31359 to create modular design functions without the need for duplication of
31360 functionality.
31361 Here’s a CommonJS module that checks user permissions:
31363 function user_context(userctx, secobj) {
31365 var is_admin = function() {
31366 return userctx.indexOf('_admin') != -1;
31367 }
31368 return {'is_admin': is_admin}
31369 }
31370 exports['user'] = user_context
31372 Each module has access to additional global variables:
31374 · module (object): Contains information about the stored module
31376 · id (string): The module id; a JSON path in ddoc context
31378 · current (code): Compiled module code object
31380 · parent (object): Parent frame
31382 · exports (object): Export statements
31384 · exports (object): Shortcut to the module.exports object
31386 The CommonJS module can be added to a design document, like so:
31388 {
31390 "views": {
31391 "lib": {
31392 "security": "function user_context(userctx, secobj) { ... }"
31393 }
31394 },
31395 "validate_doc_update": "function(newdoc, olddoc, userctx, secobj) {
31396 user = require('views/lib/security').user_context(userctx, secobj);
31397 return user.is_admin();
31398 }"
31399 "_id": "_design/test"
31400 }
31401 Modules paths are relative to the design document’s views object, but
31403 modules can only be loaded from the object referenced via lib. The lib
31404 structure can still be used for view functions as well, by simply stor‐
31405 ing view functions at e.g. views.lib.map, views.lib.reduce, etc.
31406 Erlang
31408 NOTE:
31409 The Erlang query server is disabled by default. Read configuration
31410 guide about reasons why and how to enable it.
31411 Emit(Id, Value)
31413 Emits key-value pairs to view indexer process.
31414 fun({Doc}) ->
31416 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
31417 V = proplists:get_value(<<"_id">>, Doc, null),
31418 Emit(<<K>>, V)
31419 end.
31420 FoldRows(Fun, Acc)
31422 Helper to iterate over all rows in a list function.
31423 Arguments
31425 · Fun – Function object.
31427 · Acc – The value previously returned by Fun.
31429 fun(Head, {Req}) ->
31431 Fun = fun({Row}, Acc) ->
31432 Id = couch_util:get_value(<<"id">>, Row),
31433 Send(list_to_binary(io_lib:format("Previous doc id: ~p~n", [Acc]))),
31434 Send(list_to_binary(io_lib:format("Current doc id: ~p~n", [Id]))),
31435 {ok, Id}
31436 end,
31437 FoldRows(Fun, nil),
31438 ""
31439 end.
31440 GetRow()
31442 Retrieves the next row from a related view result.
31443 %% FoldRows background implementation.
31445 %% https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=blob;f=src/couchdb/couch_native_process.erl;hb=HEAD#l368
31446 %%
31447 foldrows(GetRow, ProcRow, Acc) ->
31448 case GetRow() of
31449 nil ->
31450 {ok, Acc};
31451 Row ->
31452 case (catch ProcRow(Row, Acc)) of
31453 {ok, Acc2} ->
31454 foldrows(GetRow, ProcRow, Acc2);
31455 {stop, Acc2} ->
31456 {ok, Acc2}
31457 end
31458 end.
31459 Log(Msg)
31461 Arguments
31463 · Msg – Log a message at the INFO level.
31465 fun({Doc}) ->
31467 <<K,_/binary>> = proplists:get_value(<<"_rev">>, Doc, null),
31468 V = proplists:get_value(<<"_id">>, Doc, null),
31469 Log(lists:flatten(io_lib:format("Hello from ~s doc!", [V]))),
31470 Emit(<<K>>, V)
31471 end.
31472 After the map function has run, the following line can be found
31474 in CouchDB logs (e.g. at /var/log/couchdb/couch.log):
31475 [Sun, 04 Nov 2012 11:33:58 GMT] [info] [<0.9144.2>] Hello from 8d300b86622d67953d102165dbe99467 doc!
31477 Send(Chunk)
31479 Sends a single string Chunk in response.
31480 fun(Head, {Req}) ->
31482 Send("Hello,"),
31483 Send(" "),
31484 Send("Couch"),
31485 "!"
31486 end.
31487 The function above produces the following response:
31489 Hello, Couch!
31491 Start(Headers)
31493 Arguments
31495 · Headers – Proplist of response object.
31497 Initialize listfun response. At this point, response code and
31499 headers may be defined. For example, this function redirects to
31500 the CouchDB web site:
31501 fun(Head, {Req}) ->
31503 Start({[{<<"code">>, 302},
31504 {<<"headers">>, {[
31505 {<<"Location">>, <<"http://couchdb.apache.org">>}]
31506 }}
31507 ]}),
31508 "Relax!"
31509 end.
31510
31512 A partitioned database forms documents into logical partitions by using
31513 a partition key. All documents are assigned to a partition, and many
31514 documents are typically given the same partition key. The benefit of
31515 partitioned databases is that secondary indices can be significantly
31516 more efficient when locating matching documents since their entries are
31517 contained within their partition. This means a given secondary index
31518 read will only scan a single partition range instead of having to read
31519 from a copy of every shard.
31520 As a means to introducing partitioned databases, we’ll consider a moti‐
31522 vating use case to describe the benefits of this feature. For this
31523 example, we’ll consider a database that stores readings from a large
31524 network of soil moisture sensors.
31525 NOTE:
31527 Before reading this document you should be familiar with the theory
31528 of sharding in CouchDB.
31529 Traditionally, a document in this database may have something like the
31531 following structure:
31532 {
31534 "_id": "sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31535 "_rev":"1-14e8f3262b42498dbd5c672c9d461ff0",
31536 "sensor_id": "sensor-260",
31537 "location": [41.6171031, -93.7705674],
31538 "field_name": "Bob's Corn Field #5",
31539 "readings": [
31540 ["2019-01-21T00:00:00", 0.15],
31541 ["2019-01-21T06:00:00", 0.14],
31542 ["2019-01-21T12:00:00", 0.16],
31543 ["2019-01-21T18:00:00", 0.11]
31544 ]
31545 }
31546 NOTE:
31548 While this example uses IoT sensors, the main thing to consider is
31549 that there is a logical grouping of documents. Similar use cases
31550 might be documents grouped by user or scientific data grouped by
31551 experiment.
31552 So we’ve got a bunch of sensors, all grouped by the field they monitor
31554 along with their readouts for a given day (or other appropriate time
31555 period).
31556 Along with our documents, we might expect to have two secondary indexes
31558 for querying our database that might look something like:
31559 function(doc) {
31561 if(doc._id.indexOf("sensor-reading-") != 0) {
31562 return;
31563 }
31564 for(var r in doc.readings) {
31565 emit([doc.sensor_id, r[0]], r[1])
31566 }
31567 }
31568 and:
31570 function(doc) {
31572 if(doc._id.indexOf("sensor-reading-") != 0) {
31573 return;
31574 }
31575 emit(doc.field_name, doc.sensor_id)
31576 }
31577 With these two indexes defined, we can easily find all readings for a
31579 given sensor, or list all sensors in a given field.
31580 Unfortunately, in CouchDB, when we read from either of these indexes,
31582 it requires finding a copy of every shard and asking for any documents
31583 related to the particular sensor or field. This means that as our data‐
31584 base scales up the number of shards, every index request must perform
31585 more work, which is unnecessary since we are only interested in a small
31586 number of documents. Fortunately for you, dear reader, partitioned
31587 databases were created to solve this precise problem.
31588 What is a partition?
31590 In the previous section, we introduced a hypothetical database that
31591 contains sensor readings from an IoT field monitoring service. In this
31592 particular use case, it’s quite logical to group all documents by their
31593 sensor_id field. In this case, we would call the sensor_id the parti‐
31594 tion.
31595 A good partition has two basic properties. First, it should have a high
31597 cardinality. That is, a large partitioned database should have many
31598 more partitions than documents in any single partition. A database that
31599 has a single partition would be an anti-pattern for this feature. Sec‐
31600 ondly, the amount of data per partition should be “small”. The general
31601 recommendation is to limit individual partitions to less than ten giga‐
31602 bytes (10 GB) of data. Which, for the example of sensor documents,
31603 equates to roughly 60,000 years of data.
31604 Why use partitions?
31606 The primary benefit of using partitioned databases is for the perfor‐
31607 mance of partitioned queries. Large databases with lots of documents
31608 often have a similar pattern where there are groups of related docu‐
31609 ments that are queried together.
31610 By using partitions, we can execute queries against these individual
31612 groups of documents more efficiently by placing the entire group within
31613 a specific shard on disk. Thus, the view engine only has to consult one
31614 copy of the given shard range when executing a query instead of execut‐
31615 ing the query across all q shards in the database. This mean that you
31616 do not have to wait for all q shards to respond, which is both effi‐
31617 cient and faster.
31618 Partitions By Example
31620 To create a partitioned database, we simply need to pass a query string
31621 parameter:
31622 shell> curl -X PUT http://127.0.0.1:5984/my_new_db?partitioned=true
31624 {"ok":true}
31625 To see that our database is partitioned, we can look at the database
31627 information:
31628 shell> curl http://127.0.0.1:5984/my_new_db
31630 {
31631 "cluster": {
31632 "n": 3,
31633 "q": 8,
31634 "r": 2,
31635 "w": 2
31636 },
31637 "compact_running": false,
31638 "db_name": "my_new_db",
31639 "disk_format_version": 7,
31640 "doc_count": 0,
31641 "doc_del_count": 0,
31642 "instance_start_time": "0",
31643 "props": {
31644 "partitioned": true
31645 },
31646 "purge_seq": "0-g1AAAAFDeJzLYWBg4M...",
31647 "sizes": {
31648 "active": 0,
31649 "external": 0,
31650 "file": 66784
31651 },
31652 "update_seq": "0-g1AAAAFDeJzLYWBg4M..."
31653 }
31654 You’ll now see that the "props" member contains "partitioned": true.
31656 NOTE:
31658 Every document in a partitioned database (except _design and _local
31659 documents) must have the format “partition:docid”. More specifi‐
31660 cally, the partition for a given document is everything before the
31661 first colon. The document id is everything after the first colon,
31662 which may include more colons.
31663 NOTE:
31665 System databases (such as _users) are not allowed to be partitioned.
31666 This is due to system databases already having their own incompati‐
31667 ble requirements on document ids.
31668 Now that we’ve created a partitioned database, it’s time to add some
31670 documents. Using our earlier example, we could do this as such:
31671 shell> cat doc.json
31673 {
31674 "_id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31675 "sensor_id": "sensor-260",
31676 "location": [41.6171031, -93.7705674],
31677 "field_name": "Bob's Corn Field #5",
31678 "readings": [
31679 ["2019-01-21T00:00:00", 0.15],
31680 ["2019-01-21T06:00:00", 0.14],
31681 ["2019-01-21T12:00:00", 0.16],
31682 ["2019-01-21T18:00:00", 0.11]
31683 ]
31684 }
31685 shell> $ curl -X POST -H "Content-Type: application/json" \
31686 http://127.0.0.1:5984/my_new_db -d @doc.json
31687 {
31688 "ok": true,
31689 "id": "sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31690 "rev": "1-05ed6f7abf84250e213fcb847387f6f5"
31691 }
31692 The only change required to the first example document is that we are
31694 now including the partition name in the document id by prepending it to
31695 the old id separated by a colon.
31696 NOTE:
31698 The partition name in the document id is not magical. Internally,
31699 the database is simply using only the partition for hashing the doc‐
31700 ument to a given shard, instead of the entire document id.
31701 Working with documents in a partitioned database is no different than a
31703 non-partitioned database. All APIs are available, and existing client
31704 code will all work seamlessly.
31705 Now that we have created a document, we can get some info about the
31707 partition containing the document:
31708 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260
31710 {
31711 "db_name": "my_new_db",
31712 "doc_count": 1,
31713 "doc_del_count": 0,
31714 "partition": "sensor-260",
31715 "sizes": {
31716 "active": 244,
31717 "external": 347
31718 }
31719 }
31720 And we can also list all documents in a partition:
31722 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260/_all_docs
31724 {"total_rows": 1, "offset": 0, "rows":[
31725 {
31726 "id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31727 "key":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf",
31728 "value": {"rev": "1-05ed6f7abf84250e213fcb847387f6f5"}
31729 }
31730 ]}
31731 Note that we can use all of the normal bells and whistles available to
31733 _all_docs requests. Accessing _all_docs through the /dbname/_parti‐
31734 tion/name/_all_docs endpoint is mostly a convenience so that requests
31735 are guaranteed to be scoped to a given partition. Users are free to use
31736 the normal /dbname/_all_docs to read documents from multiple parti‐
31737 tions. Both query styles have the same performance.
31738 Next, we’ll create a design document containing our index for getting
31740 all readings from a given sensor. The map function is similar to our
31741 earlier example except we’ve accounted for the change in the document
31742 id.
31743 function(doc) {
31745 if(doc._id.indexOf(":sensor-reading-") < 0) {
31746 return;
31747 }
31748 for(var r in doc.readings) {
31749 emit([doc.sensor_id, r[0]], r[1])
31750 }
31751 }
31752 After uploading our design document, we can try out a partitioned
31754 query:
31755 shell> cat ddoc.json
31757 {
31758 "_id": "_design/sensor-readings",
31759 "views": {
31760 "by_sensor": {
31761 "map": "function(doc) { ... }"
31762 }
31763 }
31764 }
31765 shell> $ curl -X POST -H "Content-Type: application/json" http://127.0.0.1:5984/my_new_db -d @ddoc2.json
31766 {
31767 "ok": true,
31768 "id": "_design/all_sensors",
31769 "rev": "1-4a8188d80fab277fccf57bdd7154dec1"
31770 }
31771 shell> curl http://127.0.0.1:5984/my_new_db/_partition/sensor-260/_design/sensor-readings/_view/by_sensor
31772 {"total_rows":4,"offset":0,"rows":[
31773 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","0"],"value":null},
31774 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","1"],"value":null},
31775 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","2"],"value":null},
31776 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":["sensor-260","3"],"value":null}
31777 ]}
31778 Hooray! Our first partitioned query. For experienced users, that may
31780 not be the most exciting development, given that the only things that
31781 have changed are a slight tweak to the document id, and accessing views
31782 with a slightly different path. However, for anyone who likes perfor‐
31783 mance improvements, it’s actually a big deal. By knowing that the view
31784 results are all located within the provided partition name, our parti‐
31785 tioned queries now perform nearly as fast as document lookups!
31786 The last thing we’ll look at is how to query data across multiple par‐
31788 titions. For that, we’ll implement the example sensors by field query
31789 from our initial example. The map function will use the same update to
31790 account for the new document id format, but is otherwise identical to
31791 the previous version:
31792 function(doc) {
31794 if(doc._id.indexOf(":sensor-reading-") < 0) {
31795 return;
31796 }
31797 emit(doc.field_name, doc.sensor_id)
31798 }
31799 Next, we’ll create a new design doc with this function. Be sure to
31801 notice that the "options" member contains "partitioned": false.
31802 shell> cat ddoc2.json
31804 {
31805 "_id": "_design/all_sensors",
31806 "options": {
31807 "partitioned": false
31808 },
31809 "views": {
31810 "by_field": {
31811 "map": "function(doc) { ... }"
31812 }
31813 }
31814 }
31815 shell> $ curl -X POST -H "Content-Type: application/json" http://127.0.0.1:5984/my_new_db -d @ddoc2.json
31816 {
31817 "ok": true,
31818 "id": "_design/all_sensors",
31819 "rev": "1-4a8188d80fab277fccf57bdd7154dec1"
31820 }
31821 NOTE:
31823 Design documents in a partitioned database default to being parti‐
31824 tioned. Design documents that contain views for queries across mul‐
31825 tiple partitions must contain the "partitioned": false member in the
31826 "options" object.
31827 NOTE:
31829 Design documents are either partitioned or global. They cannot con‐
31830 tain a mix of partitioned and global indexes.
31831 And to see a request showing us all sensors in a field, we would use a
31833 request like:
31834 shell> curl -u adm:pass http://127.0.0.1:15984/my_new_db/_design/all_sensors/_view/by_field
31836 {"total_rows":1,"offset":0,"rows":[
31837 {"id":"sensor-260:sensor-reading-ca33c748-2d2c-4ed1-8abf-1bca4d9d03cf","key":"Bob's Corn Field #5","value":"sensor-260"}
31838 ]}
31839 Notice that we’re not using the /dbname/_partition/... path for global
31841 queries. This is because global queries, by definition, do not cover
31842 individual partitions. Other than having the “partitioned”: false
31843 parameter in the design document, global design documents and queries
31844 are identical in behavior to design documents on non-partitioned data‐
31845 bases.
31846 WARNING:
31848 To be clear, this means that global queries perform identically to
31849 queries on non-partitioned databases. Only partitioned queries on a
31850 partitioned database benefit from the performance improvements.
31851
31853 3.0.x Branch
31854 · Upgrade Notes
31855 · Version 3.0.0
31857 Upgrade Notes
31859 · #2228: The default maximum document size has been reduced to 8MB.
31860 This means that databases with larger documents will not be able to
31861 replicate into CouchDB 3.0 correctly without modification. This
31862 change has been made in preparation for anticipated hard upper limits
31863 on document size imposed by CouchDB 4.0. For 3.x, the max document
31864 size setting can be relaxed via the [couchdb] max_document_size con‐
31865 fig setting.
31866 · #2228: The default database sharding factor q has been reduced to 2
31868 by default. This, combined with automated database resharding (see
31869 below), is a better starting place for new CouchDB databases. As in
31870 CouchDB 2.x, specify ?q=# to change the value upon database creation
31871 if desired. The default can be changed via the config [cluster] q
31872 setting.
31873 · #1523, #2092, #2336, #2475: The “node-local” HTTP interface, by
31875 default exposed on port 5986, has been removed. All functionality
31876 previously available at that port is now available on the main, clus‐
31877 tered interface (by default, port 5984). Examples:
31878 GET /_node/{nodename}/_stats
31880 GET /_node/{nodename}/_system
31881 GET /_node/{nodename}/_all_dbs
31882 GET /_node/{nodename}/_uuids
31883 GET /_node/{nodename}/_config
31884 GET /_node/{nodename}/_config/couchdb/uuid
31885 POST /_node/{nodename}_config/_reload
31886 GET /_node/{nodename}/_nodes/_changes?include_docs=true
31887 PUT /_node/{nodename}/_dbs/{dbname}
31888 POST /_node/{nodename}/_restart
31889 GET /_node/{nodename}/{db-shard}
31890 GET /_node/{nodename}/{db-shard}/{doc}
31891 GET /_node/{nodename}/{db-shard}/{ddoc}/_info
31892 …and so on. Documentation has been updated to reflect this change.
31894 · #2389: CouchDB 3.0 now requires a server admin user to be defined at
31896 startup, or will print an error message and exit. (The Admin Party is
31897 now over.)
31898 [image: Dizzy the cat with a Santa hat.] [image] CC-BY-NC 2.0:
31899 hehaden @ Flickr.UNINDENT
31900 · #2576: CouchDB 3.0 now requires admin-level access for the /_all_dbs
31902 endpoint.
31903 · #2339: All databases are now created by default as admin-only. That
31905 is, the default new database _security object is now:
31906 {
31908 "members" : { "roles" : [ "_admin" ] },
31909 "admins" : { "roles" : [ "_admin" ] }
31910 }
31911 This can be changed after database creation.
31913 · Due to code changes in #2324, it is not possible to upgrade transpar‐
31915 ently from CouchDB 1.x to 3.x. In addition, the couchup utility has
31916 been removed from CouchDB 3.0 by #2399:. If you are upgrading from
31917 CouchDB 1.x, you must first upgrade to CouchDB 2.3.1 to convert your
31918 database and indexes, using couchup if desired. You can then upgrade
31919 to CouchDB 3.0. Or, you can start a new CouchDB 3.0 installation and
31920 replicate directly from 1.x to 3.0.
31921 · #1833, #2358, #1871, #1857: CouchDB 3.0 supports running only under
31923 the following Erlang/OTP versions:
31924 · 19.x - “soft” support only. No longer tested, but should work.
31926 · 20.x - must be newer than 20.3.8.11 (20.0, 20.1, 20.2 versions all
31928 invalid)
31929 · 21.x - for 21.2, must be newer than 21.2.3
31931 · 22.x - for 22.0, must be newer than 22.0.5
31933 · #1804: By default, views are limited to return a maximum of 2**28
31935 (268435456) results. This limit can be configured separately for
31936 views and partitioned views via the query_limit and parti‐
31937 tion_query_limit values in the ini file [query_server_config] sec‐
31938 tion.
31939 · After upgrading all nodes in a cluster to 3.0, add [rexi]
31941 use_kill_all = true to local.ini to save some intra-cluster network
31942 bandwidth.
31943 Deprecated feature removal
31945 The following features, deprecated in CouchDB 2.x, have been removed or
31946 replaced in CouchDB 3.0:
31947 · #2089, #2128, #2251: Local endpoints for replication targets, which
31949 never functioned as expected in CouchDB 2.x, have been completely
31950 removed. When replicating databases, always specify a full URL for
31951 the source and target. In addition, the node local _replicator data‐
31952 base is no longer automatically created.
31953 · #2163: The disk_size and data_size fields have been retired from the
31955 database info object returned by GET /{db}/. These were deprecated in
31956 CouchDB 2.x and replaced by the sizes object, which contains the
31957 improved file, active and external size metrics. Fauxton has been
31958 updated to match.
31959 · #2173: The ability to submit multiple queries against a view using
31961 the POST to /{db}/_design/{ddoc}/_view/{view} with the ?queries=
31962 option has been replaced by the new queries endpoint. The same is
31963 true of the _all_docs, _design_docs, and _local_docs endpoints.
31964 Specify a keys object when POST-ing to these endpoints.
31965 · #2248: CouchDB externals (_external/) have been removed entirely.
31967 · #2208: CouchDB no longer supports the delayed_commits option in the
31969 configuration file. All writes are now full commits. The
31970 /_ensure_full_commit API endpoint has been retained (as a no-op) for
31971 backwards compatibility with old CouchDB replicators.
31972 · #2395: The security object in the _users database cannot be edited by
31974 default. A setting exists in the configuration file to revert this
31975 behaviour. The ability to override the disable setting is expected to
31976 be removed in CouchDB 4.0.
31977 Deprecated feature warnings
31979 The following features are deprecated in CouchDB 3.0 and will be
31980 removed in CouchDB 4.0:
31981 · Show functions (/{db}/{ddoc}/_show)
31983 · List functions (/{db}/{ddoc}/_list)
31985 · Virtual hosts and ini-file rewrites
31987 · Rewrite functions (/{db}/{ddoc}/_rewrite)
31989 Version 3.0.0
31991 Features and Enhancements
31992 · #1789: User-defined partitioned databases.
31993 These special databases support user-driven placement of documents
31995 into the same shard range. JavaScript views and Mango indexes have
31996 specific optimizations for partitioned databases as well.
31997 Two tweakable configuration parameters exist:
31999 · #1842: Partition size limits. By default, each partition is limited
32001 to 10 GiB.
32002 · #1684: Partitioned database support can be disabled via feature
32004 flag in default.ini.
32005 · #1972, #2012: Automated shard splitting. Databases can now be
32007 re-sharded while online to increase the q factor to a larger number.
32008 This can be configured to require specific node and range parameters
32009 upon execution.
32010 · #1910: Automatic background indexing, internally known as ken. This
32012 subsystem ensures secondary indexes (such as JavaScript, Mango, and
32013 text search) are kept up to date, without requiring an external query
32014 to trigger building them. Many configuration parameters are avail‐
32015 able.
32016 · #1904: Completely rewritten automatic compaction daemon, internally
32018 known as smoosh. This subsystem automatically triggers background
32019 compaction jobs for both databases and views, based on configurable
32020 thresholds.
32021 · #1889, #2408: New IO Queue subsystem implementation. This is highly
32023 configurable and well-documented.
32024 · #2436, #2455: CouchDB now regression tests against, and officially
32026 supports, running on the arm64v8 (aarch64) and ppc64le (ppc64el)
32027 machine architectures. Convenience binaries are generated on these
32028 architectures for Debian 10.x (“buster”) packages, and for the Docker
32029 containers.
32030 · #1875, #2437, #2423: CouchDB now supports linking against SpiderMon‐
32032 key 60 or SpiderMonkey 1.8.5. SpiderMonkey 60 provides enhanced sup‐
32033 port for ES5, ES6, and ES2016+. Full compatibility information is
32034 available at the ECMAScript compatibility table: click on “Show obso‐
32035 lete platforms,” then look for “FF 60 ESR” in the list of engine
32036 types.
32037 However, it was discovered that on ARM 64-bit platforms, SM 60 seg‐
32039 faults frequently. Also, at the time of writing, of the set of plat‐
32040 forms supported for convenience binaries, only Debian 10.x (“buster”)
32041 ships precompiled SM 60 binaries.
32042 As a result, CouchDB’s convenience binaries only link against SM 60
32044 on debian buster ``x86_64`` and ``ppc64le`` architectures. This
32045 includes the Docker image for these architectures. All other packages
32046 and Docker images link against SM 1.8.5, as in CouchDB 2.x. As OSes
32047 update to include binaries for SM 60, the convenience binaries will
32048 be updated accordingly.
32049 · The Windows installer has many improvements, including:
32051 · Prompts for an admin user/password as CouchDB 3.0 requires * Will
32053 not overwrite existing credentials if in place
32054 · No longer remove user-modified config files, closing #1989 * Also
32056 will not overwrite them on install.
32057 · Checkbox to disable installation of the Windows service
32059 · Silent install support.
32061 · Friendly link to these online release notes in the exit dialog
32063 · Higher resolution icon for HiDPI (500x500)
32065 · #2037: Dreyfus, the CouchDB side of the Lucene-powered search solu‐
32067 tion, is now shipped with CouchDB. When one or more Clouseau Java
32068 nodes are joined to the cluster, text-based indexes can be enabled in
32069 CouchDB. It is recommended to have as many Clouseau nodes as you have
32070 CouchDB nodes. Search is advertised in the feature list present at
32071 GET / if configured correctly (#2206). Configuration and installa‐
32072 tion documentation is available.
32073 · #2411: The /_up endpoint no longer requires authentication, even when
32075 require_valid_user is true.
32076 · #2392: A new _metrics role can be given to a user. This allows that
32078 user access only to the /_node/{node}/_stats and /_node/{node}/_sys‐
32079 tem endpoints.
32080 · #1912: A new alternative systemd-journald logging backend has been
32082 added, and can be enabled through the ini file. The new backend does
32083 not include CouchDB’s microsecond-accurate timestamps, and uses the
32084 sd-daemon(3) logging levels.
32085 · #2296, #1977: If the configuration file setting [couchdb] single_node
32087 is set to true, CouchDB will automatically create the system data‐
32088 bases on startup if they are not present.
32089 · #2338, #2343: POST request to CouchDB views and the /{db}/_all_docs,
32091 /{db}/_local_docs and /{db}/_design_docs endpoints now support the
32092 same functionality as GET. Parameters are passed in the body as a
32093 JSON object, rather than in the URL when using POST.
32094 · #2292: The _scheduler/docs and _scheduler/info endpoints now return
32096 detailed replication stats for running and pending jobs.
32097 · #2282, #2272, #2290: CouchDB now supports specifying separate proxies
32099 for both the source and target in a replication via source_proxy and
32100 target_proxy keys. The API documentation has been updated.
32101 · #2240: Headers are now returned from the /{db}/_changes feed immedi‐
32103 ately, even when there are no changes available. This avoids client
32104 blocking.
32105 · #2005, #2006: The name of any node can now be retrieved through the
32107 new API endpoint GET /_node/{node-name}.
32108 · #1766: Timeouts for requests, all_docs, attachments, views, and par‐
32110 titioned view requests can all be specified separately in the ini
32111 file under the [fabric] section. See default.ini for more detail.
32112 · #1963: Metrics are now kept on the number of partition and global
32114 view queries, along with the number of timeouts that occur.
32115 · #2452, #2221: A new configuration field [couch_httpd_auth] same_site
32117 has been added to set the value of the CouchDB auth cookie’s SameSite
32118 attribute. It may be necessary to set this to strict for compatibil‐
32119 ity with future versions of Google Chrome. If CouchDB CORS support is
32120 enabled, set this to None.
32121 Performance
32123 · #2277: The couch_server process has been highly optimized, supporting
32124 significantly more load than before.
32125 · #2360: It is now possible to make the rexi interface’s unacked mes‐
32127 sage limit configurable. A new, more optimized default (5, lowered
32128 from 10) has been set. This results in a ~50% improvement on view
32129 queries on large clusters with q ≥ 8.
32130 · #2280: Connection sharing for replication now functions correctly
32132 when replicating through a forward proxy. Closes #2271.
32133 · #2195, #2207: Metrics aggregation now supports CouchDB systems that
32135 sleep or hibernate, ensuring that on wakeup does not trigger thou‐
32136 sands of unnecessary function calls.
32137 · #1795: Avoid calling fabric:update_docs with empty doc lists.
32139 · #2497: The setup wizard no longer automatically creates the
32141 _global_changes database, as the majority of users do not need this
32142 functionality. This reduces overall CouchDB load.
32143 Bugfixes
32145 · #1752, #2398, #1803: The cluster setup wizard now ensures a consis‐
32146 tent UUID and http secret across all nodes in a cluster. CouchDB
32147 admin passwords are also synced when the cluster setup wizard is
32148 used. This prevents being logged out when using Fauxton as a server
32149 admin user through a load balancer.
32150 · #2388: A compatibility change has been made to support replication
32152 with future databases containing per-document access control fields.
32153 · #2379: Any replicator error messages will provide an object in the
32155 response, or null, but never a string.
32156 · #2244, #2310: CouchDB will no longer send more data than is requested
32158 when retrieving partial attachment data blocks.
32159 · #2138: Manual operator updates to a database’s shard map will not
32161 corrupt additional database properties, such as partitioning values.
32162 · #1877: The _purge and _purged_infos_limit endpoints are now correctly
32164 restricted to server admin only.
32165 · #1794: The minimum purge sequence value for a database is now gath‐
32167 ered without a clustered _all_docs lookup.
32168 · #2351: A timeout case clause in fabric_db_info has been normalised to
32170 match other case clauses.
32171 · #1897: The /{db}/_bulk_docs endpoint now correctly catches invalid
32173 (i.e., non-hexadecimal) _rev_ values and responds with a 400 error.
32174 · #2321: CouchDB no longer requires Basic auth credentials to reach the
32176 /_session endpoint for login, even when require_valid_user is
32177 enabled.
32178 · #2295: CouchDB no longer marks a job as failed permanently if the
32180 internal doc processor crashes.
32181 · #2178: View compaction files are now removed on view cleanup.
32183 · #2179: The error message logged when CouchDB does not have a _users
32185 database is now less scary.
32186 · #2153: CouchDB no longer may return a badmatch error when querying
32188 all_docs with a passed keys array.
32189 · #2137: If search is not available, return a 400 instead of 500 status
32191 code.
32192 · #2077: Any failed fsync(2) calls are now correctly raised to avoid
32194 data corruption arising from retry attempts.
32195 · #2027: Handle epoch mismatch when duplicate UUIDs are created through
32197 invalid operator intervention.
32198 · #2019: If a database is deleted and re-created while internal cluster
32200 replication is still active, CouchDB will no longer retry to delete
32201 it continuously.
32202 · #2003, #2438: CouchDB will no longer automatically reset an index
32204 file if any attempt to read its header fails (such as when the
32205 couch_file process terminates unexpectedly). CouchDB now also han‐
32206 dles the case when a view file lacks a proper header.
32207 · #1983: Improve database “external” size calcuation to be more pre‐
32209 cise.
32210 · #1971: Correctly compare ETags using weak comparison methods to sup‐
32212 port W/ prefix added by some load balancer configurations.
32213 · #1901: Invalid revision specified for a document update will no
32215 longer result in a badarg crash.
32216 · #1845: The end_time field in /_replicate now correctly converts time
32218 to UTC.
32219 · #1824: rexi stream workers are now cleaned up when the coordinator
32221 process is killed, such as when the ddoc cache is refreshed.
32222 · #1770: Invalid database _security objects no longer return a func‐
32224 tion_clause error and stack trace.
32225 · #2412: Mango execution stats now correctly count documents read which
32227 weren’t followed by a match within a given shard.
32228 · #2393, #2143: It is now possible to override the query server envi‐
32230 ronment variables COUCHDB_QUERY_SERVER_JAVASCRIPT and
32231 COUCHDB_QUERY_SERVER_COFFEESCRIPT without overwriting the
32232 couchdb/couchdb.cmd startup scripts.
32233 · #2426, #2415: The replicator now better handles the situation where
32235 design document writes to the target fail when replicating with
32236 non-admin credentials.
32237 · #2444, #2413: Replicator error messages are now significantly
32239 improved, reducing function_clause responses.
32240 · #2454: The replication auth session plugin now ignores other cookies
32242 it may receive without logging an error.
32243 · #2458: Partitioned queries and dreyfus search functions no longer
32245 fail if there is a single failed node or rexi worker error.
32246 · #1783: Mango text indexes no longer error when given an empty selec‐
32248 tor or operators with empty arrays.
32249 · #2466: Mango text indexes no longer error if the indexed document
32251 revision no longer exists in the primary index.
32252 · #2486: The $lt, $lte, $gt, and $gte Mango operators are correctly
32254 quoted internally when used in conjunction with a text index search.
32255 · #2493: The couch_auth_cache no longer has a runaway condition in
32257 which it creates millions of monitors on the _users database.
32258 Other
32260 The 3.0.0 release also includes the following minor improvements:
32261 · #2472: CouchDB now logs the correct, clustered URI at startup (by
32263 default: port 5984.)
32264 · #2034, #2416: The path to the Fauxton installation can now be speci‐
32266 fied via the COUCHDB_FAUXTON_DOCROOT environment variable.
32267 · #2447: Replication stats are both persisted when jobs are re-created,
32269 as well as properly handled when bulk document batches are split.
32270 · #2410, #2390, #1913: Many metrics were added for Mango use, including
32272 counts of unindexed queries, invalid index queries, docs examined
32273 that do and don’t meet cluster quorum, query time, etc.
32274 · #2152, #2504: CouchDB can now be started via a symlink to the binary
32276 on UNIX-based platforms.
32277 · #1844: A new internal API has been added to write custom Erlang
32279 request-level metrics reporting plugins.
32280 · #2293, #1095: The -args_file, -config and -couch_ini parameters may
32282 now be overridden via the COUCHDB_INI_FILES environment variable on
32283 UNIX-based systems.
32284 · #2352: The remsh utility now searches for the Erlang cookie in
32286 ERL_FLAGS as well as vm.args.
32287 · #2324: All traces of the (never fully functional) view-based _changes
32289 feed have been expunged from the code base.
32290 · #2337: The md5 shim (introduced to support FIPS-compliance) is now
32292 used consistently throughout the code base.
32293 · #2270: Negative and non-integer heartbeat values now return 400 Bad
32295 Request.
32296 · #2268: When rescheduling jobs, CouchDB now stops sufficient running
32298 jobs to make room for the pending jobs.
32299 · #2186: CouchDB plugin writers have a new field in which endpoint cre‐
32301 dentials may be stashed for later use.
32302 · #2183: dev/run now supports an --extra-args flag to modify the Erlang
32304 runtime environment during development.
32305 · #2105: dev/run no longer fails on unexpected remote end connection
32307 close during cluster setup.
32308 · #2118: Improve couch_epi process replacement mechanism using map
32310 childspecs functionality in modern Erlang.
32311 · #2111: When more than MaxJobs replication jobs are defined, CouchDB
32313 now correctly handles job rotation when some jobs crash.
32314 · #2020: Fix full ring assertion in fabric stream shard replacements
32316 · #1925: Support list for docid when using couch_db:purge_docs/3.
32318 · #1642: io_priority is now set properly on view update and compaction
32320 processes.
32321 · #1865: Purge now supports >100 document IDs in a single request.
32323 · #1861: The vm.args file has improved commentary.
32325 · #1808: Pass document update type for additional checks in
32327 before_doc_update.
32328 · #1835: Module lists are no longer hardcoded in .app files.
32330 · #1798, #1933: Multiple compilation warnings were eliminated.
32332 · #1826: The couch_replicator_manager shim has been fully removed.
32334 · #1820: After restarting CouchDB, JS and Elixir tests now wait up to
32336 30s for it to be ready before timing out.
32337 · #1800: make elixir supports specifying individual tests to run with
32339 tests=.
32340 · #1805: dev/run supports --with-haproxy again.
32342 · #1774: dev/run now supports more than 3 nodes.
32344 · #1779: Refactor Elixir test suite initialization.
32346 · #1769: The Elixir test suite uses Credo for static analysis.
32348 · #1776: All Python code is now formatted using Python black.
32350 · #1786: dev/run: do not create needless dev/data/ directory.
32352 · #2482: A redundant get_ring_opts call has been removed from drey‐
32354 fus_fabric_search.
32355 · #2506: CouchDB’s release candidates no longer propagate the RC tags
32357 into each Erlang application’s version string.
32358 · #2511: recon, the Erlang diagnostic toolkit, has been added to
32360 CouchDB’s build process and ships in the release + convenience bina‐
32361 ries.
32362 · Fauxton updated to v1.2.3, which includes:
32364 · Support multiple server-generated warnings when running queries
32366 · Partitioned database support
32368 · Search index support
32370 · Remove references to deprecated dbinfo fields
32372 · Improve accessibility for screen readers
32374 · Numerous CSS fixes
32376 · Improved test cases:
32378 · Many, many test race conditions and bugs have been removed (PR list
32380 too long to include here!)
32381 · More test cases were ported to Elixir, including:
32383 · Cluster with and without quorum tests (#1812)
32385 · delayed_commits (#1796)
32387 · multiple_rows (#1958)
32389 · invalid_docids (#1968)
32391 · replication (#2090)
32393 · All attachment_* tests (#1999)
32395 · copy_doc (#2000)
32397 · attachments (#1953)
32399 · erlang_views (#2237)
32401 · auth_cache, cookie_auth, lorem*, multiple_rows, users_db, utf8 (‐
32403 #2394)
32404 · etags_head (#2464, #2469)
32406 · #2431: chttpd_purge_tests have been improved in light of CI fail‐
32408 ures.
32409 · #2432: Address flaky test failure on t_invalid_view/1.
32411 · #2363: Elixir tests now run against a single node cluster, in line
32413 with the original design of the JavaScript test suite. This is a
32414 permanent change.
32415 · #1893: Add “w:3” for lots of doc tests.
32417 · #1939, #1931: Multiple fixes to improve support in constrained CI
32419 environments.
32420 · #2346: Big-endian support for the couch_compress tests.
32422 · #2314: Do not auto-index when testing update=false in Mango.
32424 · #2141: Fix couch_views encoding test.
32426 · #2123: Timeout added for fold_docs-with_different_keys test.
32428 · #2114: EUnit tests now correctly inherit necessary environment
32430 variables.
32431 · #2122: :meck.unload() is now called automatically after every test.
32433 · #2098: Fix cpse_test_purge_replication eunit test.
32435 · #2085, #2086: Fix a flaky mem3_sync_event_listener test.
32437 · #2084: Increase timeouts on two slow btree tests.
32439 · #1960, #1961: Fix for chttpd_socket_buffer_size_test.
32441 · #1922: Tests added for shard splitting functionality.
32443 · #1869: New test added for doc reads with etag If-None-Match header.
32445 · #1831: Re-introduced cpse_test_purge_seqs test.
32447 · #1790: Reorganise couch_flag_config_tests into a proper suite.
32449 · #1785: Use devclean on elixir target for consistency of Makefile.
32451 · #2476: For testing, Triq has been replaced with PropEr as an
32453 optional dependency.
32454 · External dependency updates:
32456 · #1870: Mochiweb has been updated to 2.19.0.
32458 · #1938: Folsom has been updated to 0.8.3.
32460 · #2001: ibrowse has been updated to 4.0.1-1.
32462 · #2400: jiffy has been updated to 1.0.1.
32464 · A llama! OK, no, not really. If you got this far…thank you for read‐
32466 ing.
32467 2.3.x Branch
32469 · Upgrade Notes
32470 · Version 2.3.1
32472 · Version 2.3.0
32474 Upgrade Notes
32476 · #1602: To improve security, there have been major changes in the con‐
32477 figuration of query servers, SSL support, and HTTP global handlers:
32478 1. Query servers
32480 Query servers are NO LONGER DEFINED in the .ini files, and can no
32482 longer be altered at run-time.
32483 The JavaScript and CoffeeScript query servers continue to be
32485 enabled by default. Setup differences have been moved from
32486 default.ini to the couchdb and couchdb.cmd start scripts respec‐
32487 tively.
32488 Additional query servers can now be configured using environment
32490 variables:
32491 export COUCHDB_QUERY_SERVER_PYTHON="/path/to/python/query/server.py with args"
32493 couchdb
32494 where the last segment in the environment variable (_PYTHON)
32496 matches the usual lowercase(!) query language in the design doc
32497 language field (here, python.)
32498 Multiple query servers can be configured by using more environment
32500 variables.
32501 You can also override the default servers if you need to set com‐
32503 mand- line options (such as couchjs stack size):
32504 export COUCHDB_QUERY_SERVER_JAVASCRIPT="/path/to/couchjs /path/to/main.js -S <STACKSIZE>"
32506 couchdb
32507 2. Native Query Servers
32509 The mango query server continues to be enabled by default. The
32511 Erlang query server continues to be disabled by default. This
32512 change adds a [native_query_servers] enable_erlang_query_server =
32513 BOOL setting (defaults to false) to enable the Erlang query
32514 server.
32515 If the legacy configuration for enabling the query server is
32517 detected, that is counted as a true setting as well, so existing
32518 configurations continue to work just fine.
32519 3. SSL Support
32521 Enabling SSL support in the ini file is now easier:
32523 [ssl]
32525 enable = true
32526 If the legacy httpsd configuration is found in your ini file, this
32528 will still enable SSL support, so existing configurations do not
32529 need to be changed.
32530 4. HTTP global handlers
32532 These are no longer defined in the default.ini file, but have been
32534 moved to the couch.app context. If you need to customize your han‐
32535 dlers, you can modify the app context using a couchdb.config file
32536 as usual.
32537 · #1602: Also to improve security, the deprecated os_daemons and
32539 couch_httpd_proxy functionality has been completely removed ahead of
32540 the planned CouchDB 3.0 release. We recommend the use of OS-level
32541 daemons such as runit, sysvinit, systemd, upstart, etc. to launch and
32542 maintain OS daemons instead, and the use of a reverse proxy server in
32543 front of CouchDB (such as haproxy) to proxy access to other services
32544 or domains alongside CouchDB.
32545 · #1543: The node-local (default port 5986) /_restart endpoint has been
32547 replaced by the clustered (default port 5984) endpoint
32548 /_node/$node/_restart and /_node/_local/_restart endpoints. The
32549 node-local endpoint has been removed.
32550 · #1764: All python scripts shipped with CouchDB, including couchup and
32552 the dev/run development cluster script, now specify and require
32553 Python 3.x.
32554 · #1396: CouchDB is now compatible with Erlang 21.x.
32556 · #1680: The embedded version of rebar used to build CouchDB has been
32558 updated to the last version of rebar2 available. This assists in
32559 building on non-x86 platforms.
32560 · #1857: Refuse building with known bad versions of Erlang.
32562 Version 2.3.1
32564 Features
32565 · #1811: Add new /{db}/_sync_shards endpoint (admin-only).
32566 · #1870: Update to mochiweb 2.19.0. See also #1875.
32568 · #1857: Refuse building with known bad versions of Erlang.
32570 · #1880: Compaction: Add snooze_period_ms for finer tuning.
32572 Bugfixes
32574 · #1795: Filter out empty missing_revs results in mem3_rep.
32575 · #1384: Fix function_clause error on invalid DB _security objects.
32577 · #1841: Fix end_time field in /_replicate response.
32579 · #1860: Fix read repair in a mixed cluster environment.
32581 · #1862: Fix fabric_open_doc_revs.
32583 · #1865: Support purge requests with more than 100 doc ids.
32585 · #1867: Fix timeout in chttpd_purge_tests.
32587 · #1766: Add default fabric request timeouts.
32589 · #1810: Requests return 400 Bad Request when URL length exceeds 1460
32591 characters. See #1870 for details.
32592 · #1799: Restrict _purge to server admin.
32594 · #1874: This fixes inability to set keys with regex symbols in them.
32596 · #1901: Fix badarg crash on invalid rev for individual doc update.
32598 · #1897: Fix from_json_obj_validate crash when provided rev isn’t a
32600 valid hex.
32601 · #1803: Use the same salt for admin passwords on cluster setup.
32603 · #1053: Fix python2 compatibility for couchup.
32605 · #1905: Fix python3 compatibility for couchup.
32607 Version 2.3.0
32609 Features
32610 · (Multiple) Clustered purge is now available. This feature restores
32611 the CouchDB 1.x ability to completely remove any record of a document
32612 from a database. Conditions apply; to use the feature safely, and for
32613 full details, read the complete cluster/purging documentation.
32614 · #1658: A new config setting is available, allowing an administrator
32616 to configure an initial list of nodes that should be contacted when a
32617 node boots up. Nodes in the seedlist that are successfully reached
32618 will be added to that node’s _nodes database automatically, trigger‐
32619 ing a distributed Erlang connection and replication of the internal
32620 system databases to the new node. This can be used instead of manual
32621 config or the cluster setup wizard to bootstrap a cluster. The
32622 progress of the initial seeding of new nodes is exposed at the GET
32623 /_up endpoint.
32624 · Replication supports ipv6-only peers after updating ibrowse depen‐
32626 dency.
32627 · #1708: The UUID of the server/cluster is once again exposed in the
32629 GET / response. This was a regression from CouchDB 1.x.
32630 · #1722: Stats counts between job runs of the replicator are no longer
32632 reset on job restart.
32633 · #1195, #1742: CouchDB’s _bulk_get implementation now supports the
32635 multipart/mixed and multipart/related content types if requested,
32636 extending compatibility with third-party replication clients.
32637 Performance
32639 · #1409: CouchDB no longer forces the TCP receive buffer to a fixed
32640 size of 256KB, allowing the operating system to dynamically adjust
32641 the buffer size. This can lead to siginificantly improved network
32642 performance when transferring large attachments.
32643 · #1423: Mango selector matching now occurs at the shard level, reduc‐
32645 ing the network traffic within a cluster for a mango query.
32646 · #1423: Long running operations at the node level could exceed the
32648 inter-node timeout, leading to a fabric timeout error in the logfile
32649 and a cancellation of the task. Nodes can now ping to stop that from
32650 happening.
32651 · #1560: An optimization to how external data sizes of attachments were
32653 recorded was made.
32654 · #1586: When cleaning up outdated secondary index files, the search is
32656 limited to the index directory of a specific database.
32657 · #1593: The couch_server ETS table now has the read_concurrency option
32659 set, improving access to the global list of open database handles.
32660 · #1593: Messages to update the least-recently used (LRU) cache are not
32662 sent when the [couchdb] update_lru_on_read setting is disabled.
32663 · #1625: All nodes in a cluster now run their own rexi server.
32665 Bugfixes
32667 · #1484: _stats now correctly handles the case where a map function
32668 emits an array of integers. This bug was introduced in 2.2.0.
32669 · #1544: Certain list functions could return a render_error error
32671 intermittently.
32672 · #1550: Replicator _session support was incompatible with CouchDB
32674 installations using the require_valid_user = true setting.
32675 · #1571: Under very heavy load, it was possible that rexi_server could
32677 die in such a way that it’s never restarted, leaving a cluster with‐
32678 out the ability to issue RPC calls - effectively rendering the clus‐
32679 ter useless.
32680 · #1574: The built-in _sum reduce function has been improved to check
32682 if the objects being summed are not overflowing the view storage.
32683 Previously, there was no protection for _sum-introduced overflows.
32684 · #1582: Database creation parameters now have improved validation,
32686 giving a more readable error on invalid input.
32687 · #1588: A missing security check has been restored for the noop
32689 /db/_ensure_full_commit call to restore database validation checks.
32690 · #1591: CouchDB now creates missing shard files when accessing a data‐
32692 base if necessary. This handles the situation when, on database cre‐
32693 ation, no nodes were capable of creating any of the shard files
32694 required for that database.
32695 · #1568: CouchDB now logs a warning if a changes feed is rewound to 0.
32697 This can help diagnose problems in busy or malfunctioning clusters.
32698 · #1596: It is no longer possible that a busy couch_server, under a
32700 specific ordering and timing of events, will incorrectly track
32701 open_async messages in its mailbox.
32702 · #1601, #1654: CouchDB now logs better when an error causes it to read
32704 past the EOF of a database shard. The check for whether CouchDB is
32705 trying to read too many bytes has been correctly separated out from
32706 the error indicating it has attempted to read past the EOF.
32707 · #1613: Local nodes are now filtered out during read repair opera‐
32709 tions.
32710 · #1636: A memory leak when replicating over HTTPS and a problem occurs
32712 has been squashed.
32713 · #1635: /_replicate jobs are no longer restarted if parameters haven’t
32715 changed.
32716 · #1612: JavaScript rewrite functions now send the body of the request
32718 to the rewritten endpoint.
32719 · #1631: The replicator no longer crashes if the user has placed an
32721 invalid VDU function into one of the _replicator databases.
32722 · #1644, #1647: It is no longer possible to create illegally-named
32724 databases within the reserved system space (_ prefix.)
32725 · #1650: _bulk_get is once again operational for system databases such
32727 as _users.
32728 · #1652: Access to /_active_tasks is once again restricted to server
32730 admins only.
32731 · #1662: The couch_log application no longer crashes when new, addi‐
32733 tional information is supplied by a crashing application, or when any
32734 of its own children are restarted.
32735 · #1666: Mango could return an error that would crash the
32737 couch_query_servers application. This is no longer the case.
32738 · #1655: Configuration of ets_lru in chttpd now performs proper error
32740 checking of the specified config value.
32741 · #1667: The snappy dependency has been updated to fix a memory alloca‐
32743 tion error.
32744 · #1683: Attempting to create a local document with an invalid revision
32746 no longer throws a badarg exception. Also, when setting new_edits to
32747 false and performing a bulk write operation, local documents are no
32748 longer written into the wrong btree. Finally, it is no longer possi‐
32749 ble to create a document with an empty ID during a bulk operation
32750 with new_edits set to false.
32751 · #1721: The couchup convenience script for upgrading from CouchDB 1.x
32753 now also copies a database’s _security object on migration.
32754 · #1672: When checking the status of a view compaction immediately
32756 after starting it, the total_changes and changes_done fields are now
32757 immediately populated with valid values.
32758 · #1717: If the .ini config file is read only, an attempt to update the
32760 config through the HTTP API will now result in a proper eacces error
32761 response.
32762 · #1603: CouchDB now returns the correct total_rows result when query‐
32764 ing /{db}/_design_docs.
32765 · #1629: Internal load validation functions no longer incorrectly hold
32767 open a deleted database or its host process.
32768 · #1746: Server admins defined in the ini file accessing via HTTP API
32770 no longer result in the auth cache logging the access as a miss in
32771 the statistics.
32772 · #1607: The replicator no longer fails to re-authenticate to open a
32774 remote database when its session cookie times out due to a VDU func‐
32775 tion forbidding writes or a non-standard cookie expiration duration.
32776 · #1579: The compaction daemon no longer incorrectly only compacts a
32778 single view shard for databases with a q value greater than 1.
32779 · #1737: CouchDB 2.x now performs as well as 1.x when using a _doc_ids
32781 or _design_docs filter on a changes feed.
32782 Mango
32784 Other
32785 The 2.3.0 release also includes the following minor improvements:
32786 · Improved test cases:
32788 · The Elixir test suite has been merged. These test cases are
32790 intended to replace the aging, unmaintainable JavaScript test
32791 suite, and help reduce our dependency on Mozilla Spidermonkey
32792 1.8.5. The test suite does not yet cover all of the tests that the
32793 JS test suite does. Once it achieves full coverage, the JS test
32794 suite will be removed.
32795 · Many racy test cases improved for reliable CI runs.
32797 · The Makefile targets for list-eunit-* now work correctly on macOS.
32799 · #1732, #1733, #1736: All of the test suites run and pass on the
32801 Windows platform once again.
32802 · #1597: Off-heap messages, a new feature in Erlang 19+, can now be
32804 disabled per module if desired.
32805 · #1682: A new [feature_flags] config section exists for the purpose of
32807 enabling or disabling experimental features by CouchDB developers.
32808 · A narwhal! OK, no, not really. If you got this far…thank you for
32810 reading.
32811 2.2.x Branch
32813 · Upgrade Notes
32814 · Version 2.2.0
32816 Upgrade Notes
32818 · The minimum supported version of Erlang is now 17, not R16B03. Sup‐
32819 port for Erlang 21 is still ongoing and will be provided in a future
32820 release.
32821 · The CouchDB replication client can now use the /_session endpoint
32823 when authenticating against remote CouchDB instances, improving per‐
32824 formance since re-authorization does not have to be performed with
32825 every request. Because of this performance improvement, it is recom‐
32826 mended to increase the PBKDF2 work factor beyond the default 10 to a
32827 modern default such as 10000. This is done via the local ini file
32828 setting [couch_httpd_auth] iterations = 10000.
32829 Do not do this if an older version of CouchDB is replicating TO this
32831 instance or cluster regularly, since CouchDB < 2.2.0 must perform
32832 authentication on every request and replication performance will suf‐
32833 fer.
32834 A future version will make this increased number of iterations a
32836 default.
32837 · #820, #1032: Multiple queries can now be made at the POST
32839 /{db}/_all_docs/queries, POST /{db}/_design_docs/queries and POST
32840 /{db}/_local_docs/queries endpoints. Also, a new endpoint POST
32841 /{db}/_design/{ddoc}/_view/{view}/queries has been introduced to
32842 replace the ?queries parameter formerly provided for making multiple
32843 queries to a view. The old ?queries parameter is now deprecated and
32844 will be removed in a future release of CouchDB.
32845 · The maximum http request limit, which had been lowered in 2.1.0, has
32847 been re-raised to a 4GB limit for now. (#1446). Ongoing discussion
32848 about the path forward for future releases is available in #1200 and
32849 #1253.
32850 · #1118: The least recently used (LRU) cache of databases is now only
32852 updated on database write, not read. This has lead to significant
32853 performance enhancements on very busy clusters. To restore the previ‐
32854 ous behaviour, your local ini file can contain the block [couchdb]
32855 update_lru_on_read = true.
32856 · #1153: The CouchDB replicator can now make use of the /_session end‐
32858 point rather than relying entirely on HTTP basic authentication head‐
32859 ers. This can greatly improve replication performance. We encourage
32860 you to upgrade any nodes or clusters that regularly act as replica‐
32861 tion clients to use this new feature, which is enabled by default (‐
32862 #1462).
32863 · #1283: The [couchdb] enable_database_recovery feature, which only
32865 soft-deletes databases in response to a DELETE /{db} call, is now
32866 documented in default.ini.
32867 · #1330: CouchDB externals and OS daemons are now officially deprecated
32869 and no longer documented. Support for these features will be com‐
32870 pletely removed in a future release of CouchDB (probably 3.0.0).
32871 · #1436: CouchDB proxy authentication now uses a proper chttpd_auth
32873 module, simplifying configuration in local ini files. While this is
32874 not a backward- compatible breaking change, it is best to update your
32875 local ini files to reference the new {chttpd_auth, proxy_authentica‐
32876 tion_handler} handler rather than the couch_httpd_auth version, as
32877 couch_httpd is in the process of being deprecated completely.
32878 · #1476, #1477: The obsolete update_notification feature, which was
32880 replaced by /{db}/_changes feeds c. CouchDB 1.2, has been completely
32881 removed. This feature never worked in 2.0 for databases, only for
32882 shards, making it effectively useless.
32883 Version 2.2.0
32885 Features
32886 · Much improved documentation. Highlights include:
32887 · A complete rewrite of the sharding documentation.
32889 · Developer installation notes (INSTALL.*.rst)
32891 · Much of the content of the original CouchDB Wiki has been imported
32893 into the official docs. (The old CouchDB Wiki is in the process of
32894 being deprecated.)
32895 · Much improved Fauxton functionality. Highlights include:
32897 · Search support in the code editor
32899 · Support for relative Fauxton URLs (i.e., not always at /_utils)
32901 · Replication setup enhancements for various authentication mecha‐
32903 nisms
32904 · Fixes for IE10, IE11, and Edge (we hope…)
32906 · Resolving conflicts of design documents is now allowed
32908 · #496, COUCHDB-3287: New pluggable storage engine framework has landed
32910 in CouchDB. This internal refactor makes it possible for CouchDB to
32911 use different backends for storing the base database file itself. The
32912 refactor included a full migration of the existing “legacy” storage
32913 engine into the new framework.
32914 · #603: When creating a new database on a cluster without quorum,
32916 CouchDB will now return a 202 Accepted code if possible, indicating
32917 that at least one node has written the database record to disk, and
32918 that other nodes will be updated as they return to an online state.
32919 This replaces the former 500 internal error.
32920 · #1136, #1139: When deleting a database in a cluster without quorum,
32922 CouchDB will no longer throw a 500 error status, but a 202 as long as
32923 at least one node records the deletion, or a 200 when all nodes
32924 respond. This fix parallels the one made for #603.
32925 · #745: CouchDB no longer fails to complete replicating databases with
32927 large attachments. The fix for this issue included several related
32928 changes:
32929 · The maximum http request limit, which had been lowered in 2.1.0,
32931 has been re-raised to a 4GB limit for now. (#1446). Ongoing discus‐
32932 sion about the path forward for future releases is available in
32933 #1200 and #1253.
32934 · An update to the replicator http client that improves active socket
32936 accounting, without which CouchDB can cease to be responsive over
32937 the main http interface (#1117)
32938 · The replicator’s http client no longer performs unconditional
32940 retries on failure (#1177)
32941 · A path by which CouchDB could lose track of their RPC workers dur‐
32943 ing multipart attachment processing was removed. (#1178)
32944 · When CouchDB transmits a 413 Payload Too Large response on attach‐
32946 ment upload, it now correctly flushes the receive socket before
32947 closing the connection to avoid a TCP reset, and to give the client
32948 a better chance of parsing the 413 response. In tandem, the repli‐
32949 cator http client correctly closes its own socket after processing
32950 any 413 response. (#1234)
32951 · A fabric process to receive unchunked attachments can no longer
32953 orphan processes that leave unprocessed binaries in memory until
32954 all available memory is exhausted. (#1264).
32955 · When using CouchDB’s native SSL responder (port 6984 by default),
32957 sessions are now timed out by default after 300s. This is to work
32958 around RAM explosion in the BEAM VM when using the Erlang-native
32959 SSL libraries. (#1321
32960 · #822: A new end point api/server/dbs_info has been added to return
32962 information about a list of specified databases. This endpoint can
32963 take the place of multiple queries to /{db}.
32964 · #875, #1030: couch_peruser installations can now specify a default q
32966 value for each peruser-created database that is different from the
32967 cluster’s q value. Set this in your local ini file, under
32968 [couch_peruser] q.
32969 · #876, #1068: The couch_peruser database prefix is now configurable
32971 through your local ini file, under [couch_peruser] database_prefix.
32972 · #887: Replicator documents can now include parameters for target
32974 database creation, such as "create_target_params": {"q": "1"}. This
32975 can assist in database resharding or placement.
32976 · #977: When using COPY to copy a document, CouchDB no longer fails if
32978 the new ID includes Unicode characters.
32979 · #1095: Recognize the environment variables ARGS_FILE, SYSCONFIG_FILE,
32981 COUCHDB_ARGS_FILE and COUCHDB_SYSCONFIG_FILE to overrride where
32982 CouchDB looks for the vm.args and sys.config files at startup.
32983 · #1101, #1425: Mango can now be used to find conflicted documents in a
32985 database by adding conflicts: true to a mango selector.
32986 · #1126: When queried back after saving, replication documents no
32988 longer contain sensitive credential information (such as basic
32989 authenticataion headers).
32990 · #1203:
32992 · The compaction daemon now has a snooze period, during which it
32994 waits to start the next compaction after finishing the previous
32995 one. This value is useful in setups with many databases (e.g.
32996 with couch_peruser) or many design docs, which can cause a CPU
32997 spike every check_interval seconds. The setting can be adjusted
32998 in your local ini file via [compaction_daemon] snooze_period.
32999 The current default is a 3 second pause.
33000 · The check_interval has been raised from 300 seconds to 3600 sec‐
33002 onds.
33003 · A notice-level log about closing view indexes has been demoted
33005 to the debug level. In a sceario with many design docs, this
33006 would createsignficant load on the logging subsystem every [com‐
33007 paction_daemon] check_interval for no discernible benefit.
33008 · #1309, #1435: CouchDB now reports the git sha at the time of build in
33010 the top-level GET / version string, in a new git_sha key. This can be
33011 used to help ensure an unmodified version of CouchDB has been built
33012 and is running on any given machine.
33013 · COUCHDB-2971, #1346: CouchDB now includes a new builtin reduce func‐
33015 tion _approx_count_distinct, that uses a HyperLogLog algorithm to
33016 estimate the number of distinct keys in the view index. The precision
33017 is currently fixed to 2^11 observables, and therefore uses approxi‐
33018 mately 1.5KB of memory.
33019 · #1377: CouchDB finalization of view reduces now occurs at the coordi‐
33021 nator node. This simplified the built-in _stats function.
33022 · #1392: When running CouchDB under Erlang 19.0 or newer, messages can
33024 now be stored off the process heap. This is extremely useful for
33025 Erlang processes that can have huge number of messages in their mail‐
33026 box, and is now enabled for couch_server, couch_log_server,
33027 ddoc_cache, mem3_shards, and rexi_server whenever possible.
33028 · #1424: The CouchDB native SSL/TLS server httpsd now accepts
33030 socket-level configuration options through the [httpsd]
33031 server_options ini file setting.
33032 · #1440: CouchDB can now be configured to prevent non-admins from
33034 accessing the GET /_all_dbs method by specifying [chttpd]
33035 admin_only_all_dbs = true in your local ini file(s). The true setting
33036 will become default in future versions.
33037 · #1171, #1445: CouchDB can now be configured to use the internal
33039 Erlang MD5 hash function when not available in the external environ‐
33040 ment (e.g. FIPS enabled CentOS) at compile time with the configure
33041 flag --enable-md5. Because this implementation is slower, it is not
33042 recommended in the general case.
33043 Performance
33045 · #958: The revision stemming algorithm was optimized down from O(N^2)
33046 to O(N) via a depth-first search approach, and then further improved
33047 by calling the stemming operation only when necessary. This new algo‐
33048 rithm can be disabled by setting the option [couchdb] stem_interac‐
33049 tive_updates = false if necessary.
33050 · #1246: CouchDB now checks for request authorization only once per
33052 each database request, improving the performance of any request that
33053 requires authorization.
33054 Bugfixes
33056 · #832, #1064: Tracking of Couch logging stats has been added back into
33057 the per-node /_node/<node-name>/_stats endpoint.
33058 · #953, #973: Return 404 Not Found on GET /_scheduler, not 405 Method
33060 Not Allowed.
33061 · #955: The /{db}/_bulk_docs endpoint now correctly responds with a 400
33063 Bad Request error if the new_edits parameter is not a boolean.
33064 · #969: CouchDB now returns offset and update_seq values when keys are
33066 provided to the GET or POST /{db}/_all_docs?update_seq=true end‐
33067 points. This was affecting PouchDB compatibility.
33068 · #984, #1434: CouchDB views now retain their update_seq after com‐
33070 paction, preventing potentially expensive client-side view rewinds
33071 after compaction.
33072 · #1012: Address a theoretical race condition the replication scheduler
33074 could encounter when trying to determine if the cluster is “stable”
33075 enough to resume handling replication-introduced document updates.
33076 · #1051: Return a user-friendly error message when attempting to create
33078 a CouchDB user with an invalid password field (non-string).
33079 · #1059: DB-specific compaction configurations were not working cor‐
33081 rectly. The syntax now also supports shard-level custom compaction
33082 configuration if desired (which it probably isn’t.)
33083 · #1097: Compaction daemon will not crash out when trying to check spe‐
33085 cific file system mounts that are not “real” file systems (like /run
33086 on Linux).
33087 · #1198: Fauxton is no longer available on the node-local port (5986,
33089 by default). The node-local port is only to be used for specific
33090 administrative tasks; removing the Fauxton interface prevents mistak‐
33091 ing the node-local port as the correct CouchDB port (5984, by
33092 default).
33093 · #1165: validate_doc_update view functions can once again be imple‐
33095 mented directly in Erlang (after enabling the optional Erlang view
33096 server).
33097 · #1223: The couch_config application now correctly handles non-persis‐
33099 tent integer and boolean-valued configuration changes.
33100 · #1242: couch_os_daemons may now reside in directories with spaces.
33102 · #1258: CouchDB will now successfully login users, even if password
33104 encryption is very slow.
33105 · #1276: The replication scheduler status for a repeatedly erroring job
33107 now correctly reflects the crashing state in more scenarios.
33108 · #1375: If CouchDB fails authorization but passes authentication, it
33110 no longer drops the user_ctx out of the request.
33111 · #1390: The active size of views (as returned in a database info
33113 response) no longer is incorrectly calculated in such a way that it
33114 could occasionally be larger than the actual on-disk file size.
33115 · #1401: CouchDB Erlang views no longer crash in the couch_native
33117 process with an unexpected function_clause error.
33118 · #1419: When deleting a file, CouchDB now properly ignores the config‐
33120 uration flag enable_database_recovery when set when compacting data‐
33121 bases, rather than always retaining the old, renamed, uncompacted
33122 database file.
33123 · #1439: The CouchDB setup wizard now correctly validates
33125 bind_addresses. It also no longer logs credentials by moving logging
33126 of internal wizard setup steps to the debug level from the notice
33127 level.
33128 Mango
33130 · #816, #962, #1038: If a user specifies a value for use_index that is
33131 not valid for the selector (does not meet coverage requirements or
33132 proper sort fields), attempt to fall back to a valid index or full DB
33133 scan rather than returning a 400. If we fall back, populate a warn‐
33134 ing field in the response. Mango also tries to use indexes where $or
33135 may select a field only when certain values are present.
33136 · #849: When {"seq_indexed": true} is specified, a badmatch error was
33138 returned. This is now fixed.
33139 · #927, #1310: Error messages when attempting to sort incorrectly are
33141 now actually useful.
33142 · #951: When using GET /{db}/_index, only use a partial filter selector
33144 for an index if it is set to something other than the default.
33145 · #961: Do not prefix _design/ to a Mango index name whose user-speci‐
33147 fied name already starts with _design/.
33148 · #988, #989: When specifying a use_index value with an invalid index,
33150 correctly return a 400 Bad Request showing that the requested index
33151 is invalid for the request specified.
33152 · #998: The fix for CVE 2017-12635 presented a breaking change to
33154 Mango’s /{db}/_find, which would evaluate all instances of all JSON
33155 fields in a selector. Mango is now tested to ensure it only considers
33156 the last instance of a field, silently ignoring those that appear
33157 before it.
33158 · #1014: Correctly deduce list of indexed fields in a selector when
33160 nested $and operators are specified.
33161 · #1023: Fix an unexpected 500 error if startkey and endkey in a Mango
33163 selector were reversed.
33164 · #1067: Prevent an invalid_cast crash when the couch_proc_manager soft
33166 limit for processes is reached and mango idle processes are stopped.
33167 · #1336: The built-in fields _id and rev will always be covered by any
33169 index, and Mango now correctly ignores their presence in any index
33170 that explicitly includes them for selector matching purposes.
33171 · #1376: Mango now appropriately selects some indexes as usable for
33173 queries, even if not all columns for an index are added to the
33174 query’s sort field list.
33175 · Multiple fixes related to using Mango as a front-end for full text
33177 indexing (a feature not shipped with couch, but for which support is
33178 in place as a compile-time addon).
33179 Other
33181 The 2.2.0 release also includes the following minor improvements:
33182 · Developers can, at build time, enable curl libraries & disable Faux‐
33184 ton and documentation builds by specifying the new --dev option to
33185 the configure script.
33186 · The mochiweb dependency was bumped to version 2.17.0, in part to
33188 address the difficult #745 issue.
33189 · Improved compatibility with newer versions of Erlang (20.x)
33191 · Improved release process for CouchDB maintainers and PMC members.
33193 · Multiple test suite improvements, focused on increased coverage,
33195 speed, and reliability.
33196 · Improvements to the Travis CI and Jenkins CI setups, focused on
33198 improved long-term project maintenance and automatability.
33199 · Related improvements to the CouchDB deb/rpm packaging and Docker
33201 repositories to make deployment even easier.
33202 · #1007: Move etc/default.ini entries back into [replicator] section
33204 (incorrectly moved to [couch_peruser] section)
33205 · #1245: Increased debug-level logging for shard open errors is now
33207 available.
33208 · #1296: CouchDB by default now always invokes the SMP-enabled BEAM VM,
33210 even on single-processor machines. A future release of Erlang will
33211 remove the non-SMP BEAM VM entirely.
33212 · A pony! OK, no, not really. If you got this far…thank you for read‐
33214 ing.
33215 2.1.x Branch
33217 · Upgrade Notes
33218 · Version 2.1.2
33220 · Version 2.1.1
33222 · Version 2.1.0
33224 · Fixed Issues
33226 Upgrade Notes
33228 · When upgrading from 2.x to 2.1.1, if you have not customized your
33229 node name in vm.args, be sure to retain your original vm.args file.
33230 The default node name has changed from couchdb@localhost to
33231 couchdb@127.0.0.1, which can prevent CouchDB from accessing existing
33232 databases on the system. You may also change the name option back to
33233 the old value by setting -name couchdb@localhost in etc/vm.args by
33234 hand. The default has changed to meet new guidelines and to provide
33235 additional functionality in the future.
33236 If you receive errors in the logfile, such as internal_server_error :
33238 No DB shards could be opened. or in Fauxton, such as This database
33239 failed to load. you need to make this change.
33240 · The deprecated (and broken) OAuth 1.0 implementation has been
33242 removed.
33243 · If user code reads or manipulates replicator document states, con‐
33245 sider using the [replicator] update_docs = true compatibility parame‐
33246 ter. In that case the replicator will continue updating documents
33247 with transient replication states. However, that will incur a perfor‐
33248 mance cost. Consider instead using the _scheduler/docs HTTP endpoint.
33249 · The stale parameter for views and _find has been deprecated in favour
33251 of two new parameters: stable and update. The old stale=ok behaviour
33252 is equivalent to stable=true&update=false, and the old
33253 stale=update_after behaviour is equivalent to sta‐
33254 ble=true&update=lazy. The deprecated stale parameter will be removed
33255 in CouchDB 3.0.
33256 · The new httpd/max_http_request_size configuration parameter was
33258 added. This has the same behavior as the old couchdb/max_docu‐
33259 ment_size configuration parameter, which had been unfortunately mis‐
33260 named, and has now been updated to behave as the name would suggest.
33261 Both are documented in the shipped default.ini file.
33262 Note that the default for this new parameter is 64MB instead of 4GB.
33264 If you get errors when trying to PUT or POST and see HTTP 413 return
33265 codes in couchdb logs, this could be the culprit. This can affect
33266 couchup in-place upgrades as well.
33267 · #914: Certain critical config sections are blacklisted from being
33269 modified through the HTTP API. These sections can still be modified
33270 through the standard local.ini or local.d/*.ini files.
33271 · #916: couchjs now disables eval() and the Function() constructor by
33273 default. To restore the original behaviour, add the --eval flag to
33274 the definition of the javascript query server in your local.ini file.
33275 Version 2.1.2
33277 Security
33278 · CVE 2018-8007
33279 Version 2.1.1
33281 Security
33282 · CVE 2017-12635
33283 · CVE 2017-12636
33285 General
33287 · #617: CouchDB now supports compilation and running under Erlang/OTP
33288 20.x.
33289 · #756: The couch_peruser functionality is now really fixed. Really.
33291 · #827: The cookie domain for AuthSession cookies, used in a proxy
33293 authentication configuration, can now be customized via the ini file.
33294 · #858: It is now possible to modify shard maps for system databases.
33296 · #732: Due to an Erlang bug (ERL-343), invalid paths can be returned
33298 if volumes are mounted containing whitespace in their name. This
33299 problem surfaced primarily on macOS (Time Machine volumes). CouchDB
33300 now works around this bug in unpatched versions of Erlang by skipping
33301 the free space check performed by the compaction daemon. Erlang
33302 itself will correctly perform free space checks in version 21.0.
33303 · #824: The current node’s local interface can now be accessed at
33305 /_node/_local/{endpoint} as well as at /_node/<nodename>@<host‐
33306 name>/{endpoint}.
33307 · The Dockerfile in the source repository has been retired. For a cur‐
33309 rent Dockerfile, see the couchdb-docker repository.
33310 · Fauxton now uses a version of React with a BSD license.
33312 Performance
33314 · #835: CouchDB now no longer decompresses documents just to determine
33315 their uncompressed size. In tests, this has lead to improvements
33316 between 10-40% in both CPU and wall-clock time for database com‐
33317 paction.
33318 · The design document cache (ddoc_cache) has been rewritten to improve
33320 performance.
33321 Mango
33323 · #808: Mango now supports partial indexes. Partial indexes allow docu‐
33324 ments to be filtered at indexing time, potentially offering signifi‐
33325 cant performance improvements for query selectors that don’t map
33326 cleanly to a range query on an index.
33327 · #740: Mango queries can now be paginated. Each query response
33329 includes a bookmark. The bookmark can be provided on a subsequent
33330 query to continue from a specific key.
33331 · #768: Mango _find accepts an execution_stats parameter. If present, a
33333 new object is included in the response which contains information
33334 about the query executed. The object contains the count of total keys
33335 examined (0 for json indexes), total documents examined (when
33336 include_docs=true is used), and the total quorum documents examined
33337 (when fabric doc lookups are used).
33338 · #816 and #866: Mango now requires that all of the fields in a candi‐
33340 date index must exist in a query’s selector. Previously, this check
33341 was incorrect, and indexes that might only contain a subset of valid
33342 documents might be selected by the query planner if no explicit index
33343 was specified at query time. Further, if a sort field is specified at
33344 query time, that field needs to exist (but could be null) in the
33345 results returned.
33346 Other
33348 The 2.1.1 release also includes the following minor improvements:
33349 · #635: Stop couch_index processes on ddoc update
33351 · #721: Save migrated replicator checkpoint documents immediately
33353 · #688: Reuse http-based replication checkpoints when upgrading to
33355 https
33356 · #729: Recommend the use only of -name and not -sname in vm.args
33358 for compatibility.
33359 · #738: Allow replicator application to always update replicator
33361 docs.
33362 · #605: Add Prefer: return=minimal header options from RFC7240 to
33364 reduce the number of headers in the response.
33365 · #744: Allow a 503 response to be returned to clients (with metric
33367 support)
33368 · #746: Log additional information on crashes from rexi
33370 · #752: Allow Mango $in queries without requiring the index to use
33372 an array
33373 · (multiple) Additional debugging utilities have been added.
33375 · (multiple) Hot code upgrades from 2.0 -> 2.1.1 are now possible.
33377 · (multiple) Improvements to the test suite have been made.
33379 · #765: Mango _explain now includes view parameters as requested by
33381 the user.
33382 · #653: _show and _list should now work for admin-only databases
33384 such as _users.
33385 · #807: Mango index selection should occur only once.
33387 · #804: Unhandled Mango errors are now logged.
33389 · #659: Improve accuracy of the max_document_size check.
33391 · #817: Invalid Base64 in inline attachments is now caught.
33393 · #825: Replication IDs no longer need to be URL encoded when using
33395 the _scheduler/jobs/<job_id> endpoint.
33396 · #838: Do not buffer rexi messages to disconnected nodes.
33398 · #830: The stats collection interval is now configurable in an ini
33400 file, not in the application context. The default value is 10, and
33401 the setting is reloaded every 600 seconds.
33402 · #812: The /{db} endpoint now includes a cluster block with the
33404 database’s q, n, and default w and r values. This supplements the
33405 existing /{db}/_shards and /{db}/_shards/{id} detailed information
33406 on sharding and quorum.
33407 · #810: The replicator scheduler crashed counter gauge more reliably
33409 detects replication crashes by reducing the default number of
33410 retries from 10 to 5 (reducing the duration from 4 mins to 8
33411 secs).
33412 · COUCHDB-3288: Tolerate mixed clusters for the upcoming pluggable
33414 storage engine work.
33415 · #839: Mango python tests now support Python 3 as well as 2.
33417 · #845: A convenience remsh script has been added to support live
33419 debugging of running systems.
33420 · #846: Replicator logging is now less verbose and more informative
33422 when replication terminates unexpectedly.
33423 · #797: Reduce overflow errors are now returned to the client,
33425 allowing views with a single bad reduce to build while not
33426 exhausting the server’s RAM usage.
33427 · #881: Mango now allows match on documents where the indexed value
33429 is an object if a range query is issued. Previously, query results
33430 might change in the presence of an index, and operators/selectors
33431 which explicitly depend on a full index scan (such as $exists)
33432 would not return a complete result set.
33433 · #883: Erlang time module compatibility has been improved for
33435 releases of Erlang newer than 18.0.
33436 · #933: 410 is now returned when attempting to make a temporary view
33438 request.
33439 · #934: The replicator now has a configurable delay before retrying
33441 to retrieve a document after receiving a missing_doc error.
33442 · #936: jiffy now deduplicates JSON keys.
33444 Version 2.1.0
33446 · The Mango _find endpoint supports a new combination operator, $all‐
33447 Match, which matches and returns all documents that contain an array
33448 field with all its elements matching all the specified query crite‐
33449 ria.
33450 · New scheduling replicator. The core of the new replicator is a sched‐
33452 uler which allows running a large number of replication jobs by
33453 switching between them, stopping some and starting others periodi‐
33454 cally. Jobs which fail are backed off exponentially. There is also an
33455 improved inspection and querying API: _scheduler/jobs and _sched‐
33456 uler/docs:
33457 · _scheduler/jobs : This endpoint shows active replication jobs.
33459 These are jobs managed by the scheduler. Some of them might be run‐
33460 ning, some might be waiting to run, or backed off (penalized)
33461 because they crashed too many times. Semantically this is somewhat
33462 equivalent to _active_tasks but focuses only on replications. Jobs
33463 which have completed or which were never created because of mal‐
33464 formed replication documents will not be shown here as they are not
33465 managed by the scheduler. _replicate replications, started form
33466 _replicate endpoint not from a document in a _replicator db, will
33467 also show up here.
33468 · _scheduler/docs : This endpoint is an improvement on having to go
33470 back and read replication documents to query their state. It repre‐
33471 sents the state of all the replications started from documents in
33472 _replicator db. Unlike _scheduler/jobs it will also show jobs which
33473 have failed or have completed.
33474 By default, scheduling replicator will not update documents with
33476 transient states like triggered or error anymore, instead _sched‐
33477 uler/docs API should be used to query replication document states.
33478 Other scheduling replicator improvements
33480 · Network resource usage and performance was improved by implement‐
33481 ing a shared connection pool. This should help in cases of a large
33482 number of connections to the same sources or target. Previously
33483 connection pools were shared only withing a single replication
33484 job.
33485 · Improved request rate limit handling. Replicator requests will
33487 auto-discover rate limit capacity on targets and sources based on
33488 a proven Additive Increase / Multiplicative Decrease feedback con‐
33489 trol algorithm.
33490 · Improved performance by having exponential backoff for all repli‐
33492 cation jobs failures. Previously there were some scenarios were
33493 failure led to continuous repeated retries, consuming CPU and disk
33494 resources in the process.
33495 · Improved recovery from long but temporary network failure. Cur‐
33497 rently if replications jobs fail to start 10 times in a row, they
33498 will not be retried anymore. This is sometimes desirable, but in
33499 some cases, for example, after a sustained DNS failure which even‐
33500 tually recovers, replications reach their retry limit, stop retry‐
33501 ing and never recover. Previously it required user intervention to
33502 continue. Scheduling replicator will never give up retrying a
33503 valid scheduled replication job and so it should recover automati‐
33504 cally.
33505 · Better handling of filtered replications. Failing user filter code
33507 fetches from the source will not block replicator manager and
33508 stall other replications. Failing filter fetches will also be
33509 backed off exponentially. Another improvement is when filter code
33510 changes on the source, a running replication will detect that and
33511 restart itself with a new replication ID automatically.
33512 The 2.1.0 release also includes the following minor improvements:
33514 · COUCHDB-1946: Hibernate couch_stream after each write (up to 70%
33516 reduction in memory usage during replication of DBs with large
33517 attachments)
33518 · COUCHDB-2964: Investigate switching replicator manager change
33520 feeds to using “normal” instead of “longpoll”
33521 · COUCHDB-2988: (mango) Allow query selector as changes and replica‐
33523 tion filter
33524 · COUCHDB-2992: Add additional support for document size
33526 · COUCHDB-3046: Improve reduce function overflow protection
33528 · COUCHDB-3061: Use vectored reads to search for buried headers in
33530 .couch files. “On a modern linux system with SSD, we see improve‐
33531 ments up to 15x.”
33532 · COUCHDB-3063: “stale=ok” option replaced with new “stable” and
33534 “update” options.
33535 · COUCHDB-3180: Add features list in the welcome message
33537 · COUCHDB-3203: Make auth handlers configurable (in ini files)
33539 · COUCHDB-3234: Track open shard timeouts with a counter instead of
33541 logging
33542 · COUCHDB-3242: Make get view group info timeout in couch_indexer
33544 configurable
33545 · COUCHDB-3249: Add config to disable index all fields (text
33547 indexes)
33548 · COUCHDB-3251: Remove hot loop usage of filename:rootname/1
33550 · COUCHDB-3284: 8Kb read-ahead in couch_file causes extra IO and
33552 binary memory usage
33553 · COUCHDB-3298: Optimize writing btree nodes
33555 · COUCHDB-3302: (Improve) Attachment replication over low bandwidth
33557 network connections
33558 · COUCHDB-3307: Limit calls to maybe_add_sys_db_callbacks to once
33560 per db open
33561 · COUCHDB-3318: bypass couch_httpd_vhost if there are none
33563 · COUCHDB-3323: Idle dbs cause excessive overhead
33565 · COUCHDB-3324: Introduce couch_replicator_scheduler
33567 · COUCHDB-3337: End-point _local_docs doesn’t conform to query
33569 params of _all_docs
33570 · COUCHDB-3358: (mango) Use efficient set storage for field names
33572 · COUCHDB-3425: Make _doc_ids _changes filter fast-path limit con‐
33574 figurable
33575 · #457: TeX/LaTeX/texinfo removed from default docs build chain
33577 · #469: (mango) Choose index based on fields match
33579 · #483: couchup database migration tool
33581 · #582: Add X-Frame-Options support to help protect against click‐
33583 jacking
33584 · #593: Allow bind address of 127.0.0.1 in _cluster_setup for single
33586 nodes
33587 · #624: Enable compaction daemon by default
33589 · #626: Allow enable node decom using string “true”
33591 · (mango) Configurable default limit, defaults to 25.
33593 · (mango) _design documents ignored when querying _all_docs
33595 · (mango) add $allMatch selector
33597 · Add local.d/default.d directories by default and document
33599 · Improved INSTALL.* text files
33601 Fixed Issues
33603 The 2.1.0 release includes fixes for the following issues:
33604 · COUCHDB-1447: X-Couch-Update-NewRev header is missed if custom head‐
33606 ers are specified in response of _update handler (missed in 2.0
33607 merge)
33608 · COUCHDB-2731: Authentication DB was not considered a system DB
33610 · COUCHDB-3010: (Superceded fix for replication exponential backoff)
33612 · COUCHDB-3090: Error when handling empty “Access-Control-Request-Head‐
33614 ers” header
33615 · COUCHDB-3100: Fix documentation on require_valid_user
33617 · COUCHDB-3109: 500 when include_docs=true for linked documents
33619 · COUCHDB-3113: fabric:open_revs can return {ok, []}
33621 · COUCHDB-3149: Exception written to the log if db deleted while there
33623 is a change feed running
33624 · COUCHDB-3150: Update all shards with stale=update_after
33626 · COUCHDB-3158: Fix a crash when connection closes for _update
33628 · COUCHDB-3162: Default ssl settings cause a crash
33630 · COUCHDB-3164: Request fails when using
33632 _changes?feed=eventsource&heartbeat=30000
33633 · COUCHDB-3168: Replicator doesn’t handle well writing documents to a
33635 target db which has a small max_document_size
33636 · COUCHDB-3173: Views return corrupt data for text fields containing
33638 non-BMP characters
33639 · COUCHDB-3174: max_document_size setting can by bypassed by issuing
33641 multipart/related requests
33642 · COUCHDB-3178: Fabric does not send message when filtering lots of
33644 documents
33645 · COUCHDB-3181: function_clause error when adding attachment to doc in
33647 _users db
33648 · COUCHDB-3184: couch_mrview_compactor:recompact/1 does not handle
33650 errors in spawned process
33651 · COUCHDB-3193: fabric:open_revs returns multiple results when one of
33653 the shards has stem_interactive_updates=false
33654 · COUCHDB-3199: Replicator VDU function doesn’t acount for an already
33656 malformed document in replicator db
33657 · COUCHDB-3202: (mango) do not allow empty field names
33659 · COUCHDB-3220: Handle timeout in _revs_diff
33661 · COUCHDB-3222: (Fix) HTTP code 500 instead of 400 for invalid key dur‐
33663 ing document creation
33664 · COUCHDB-3231: Allow fixing users’ documents (type and roles)
33666 · COUCHDB-3232: user context not passed down in fabric_view_all_docs
33668 · COUCHDB-3238: os_process_limit documentation wrong
33670 · COUCHDB-3241: race condition in couch_server if delete msg for a db
33672 is received before open_result msg
33673 · COUCHDB-3245: Make couchjs -S option take effect again
33675 · COUCHDB-3252: Include main-coffee.js in release artifact (broken Cof‐
33677 feeScript view server)
33678 · COUCHDB-3255: Conflicts introduced by recreating docs with attach‐
33680 ments
33681 · COUCHDB-3259: Don’t trap exits in couch_file
33683 · COUCHDB-3264: POST to _all_docs does not respect conflicts=true
33685 · COUCHDB-3269: view response can ‘hang’ with filter and limit speci‐
33687 fied
33688 · COUCHDB-3271: Replications crash with ‘kaboom’ exit
33690 · COUCHDB-3274: eof in couch_file can be incorrect after error
33692 · COUCHDB-3277: Replication manager crashes when it finds _replicator
33694 db shards which are not part of a mem3 db
33695 · COUCHDB-3286: Validation function throwing unexpected json crashes
33697 with function_clause
33698 · COUCHDB-3289: handle error clause when calling fabric:open_revs
33700 · COUCHDB-3291: Excessively long document IDs prevent replicator from
33702 making progress
33703 · COUCHDB-3293: Allow limiting length of document ID (for CouchDB
33705 proper)
33706 · COUCHDB-3305: (mango) don’t crash with invalid input to built in
33708 reducer function
33709 · COUCHDB-3362: DELETE attachment on non-existing document creates the
33711 document, rather than returning 404
33712 · COUCHDB-3364: Don’t crash compactor when compacting process fails.
33714 · COUCHDB-3367: Require server admin user for db/_compact and
33716 db_view_cleanup endpoints
33717 · COUCHDB-3376: Fix mem3_shards under load
33719 · COUCHDB-3378: Fix mango full text detection
33721 · COUCHDB-3379: Fix couch_auth_cache reinitialization logic
33723 · COUCHDB-3400: Notify couch_index_processes on all shards when ddoc
33725 updated
33726 · COUCHDB-3402: race condition in mem3 startup
33728 · #511: (mango) Return false for empty list
33730 · #595: Return 409 to PUT attachment with non-existent rev
33732 · #623: Ensure replicator _active_tasks entry reports recent pending
33734 changes value
33735 · #627: Pass UserCtx to fabric’s all_docs from mango query
33737 · #631: fix couchdb_os_proc_pool eunit timeouts
33739 · #644: Make couch_event_sup:stop/1 synchronous
33741 · #645: Pass db open options to fabric_view_map for _view and _list
33743 queries on _users DB
33744 · #648: Fix couch_replicator_changes_reader:process_change
33746 · #649: Avoid a race when restarting an index updater
33748 · #667: Prevent a terrible race condition
33750 · #677: Make replication filter fetch error for _replicate return a 404
33752 · Fix CORS max_age configuration parameter via Access-Control-Max-Age
33754 · Chunk missing revisions before attempting to save on target (improves
33756 replication for very conflicted, very deep revision tree documents)
33757 · Allow w parameter for attachments
33759 · Return “Bad Request” when count in /_uuids exceeds max
33761 · Fix crashes when replicator db is deleted
33763 · Skip internal replication if changes already replicated
33765 · Fix encoding issues on _update/../doc_id and PUT attachments
33767 2.0.x Branch
33769 · Version 2.0.0
33770 · Upgrade Notes
33772 · Known Issues
33774 · Breaking Changes
33776 Version 2.0.0
33778 · Native clustering is now supported. Rather than use CouchDB replica‐
33779 tion between multiple, distinct CouchDB servers, configure a cluster
33780 of CouchDB nodes. These nodes will use an optimized Erlang-driven
33781 ‘internal replication’ to ensure data durability and accessibility.
33782 Combine a clustered CouchDB with a load balancer (such as haproxy) to
33783 scale CouchDB out horizontally. More details of the clustering fea‐
33784 ture are available in the cluster.
33785 · Futon replaced by brand-new, completely re-engineered Fauxton inter‐
33787 face. URL remains the same.
33788 · The new Mango Query Server provides a simple JSON-based way to per‐
33790 form CouchDB queries without JavaScript or MapReduce. Mango Queries
33791 have a similar indexing speed advantage over JavaScript Queries than
33792 the Erlang Queries have (2x-10x faster indexing depending on doc size
33793 and system configuration). We recommend all new apps start using
33794 Mango as a default. Further details are available in the _find,
33795 _index and _explain API.
33796 · Mango selectors can be used in _changes feeds instead of JavaScript
33798 MapReduce filters. Mango has been tested to be up to an order of mag‐
33799 nitude (10x) faster than JavaScript in this application.
33800 · Rewrite rules for URLs can be performed using JavaScript functions.
33802 · Multiple queries can be made of a view with a single HTTP request.
33804 · Views can be queried with sorting turned off ( sorted=false) for a
33806 performance boost.
33807 · The global changes feed has been enhanced. It is now resumable and
33809 persistent.
33810 · New endpoints added (documentation forthcoming):
33812 · api/server/membership shows all nodes in a cluster
33814 · /_bulk_get speeds up the replication protocol over low-latency con‐
33816 nections
33817 · /_node/ api to access individual nodes’ configuration and com‐
33819 paction features
33820 · /_cluster_setup api to set up a cluster from scratch.
33822 · /_up api to signal health of a node to a load-balancer
33824 · /db/_local_docs and /db/_design_docs (similar to /db/_all_docs)
33826 · The /_log endpoint was removed.
33828 · “Backend” interface on port 5986 used for specific cluster admin
33830 tasks. Of interest are the _nodes and _dbs databases visible only
33831 through this interface.
33832 · Support added for Erlang/OTP 17.x, 18.x and 19
33834 · New streamlined build system written for Unix-like systems and Micro‐
33836 soft Windows
33837 · Configuration has moved from /_config to /_node/{node-name}/_config
33839 · instance_start_time now always reports "0".
33841 Upgrade Notes
33843 · The update sequences returned by the api/db/changes feed are no
33844 longer integers. They can be any JSON value. Applications should
33845 treat them as opaque values and return them to CouchDB as-is.
33846 · Temporary views are no longer supported.
33848 · It is possible to have multiple replicator databases. replicator/db
33850 config option has been removed. Instead _replicator and any database
33851 names ending with the /_replicator suffix will be recognized as
33852 replicator databases by the system.
33853 · Note that the semantics of some API calls have changed due to the
33855 introduction of the clustering feature. Specifically, make note of
33856 the difference between receiving a 201 and a 202 when storing a docu‐
33857 ment.
33858 · all_or_nothing is no longer supported by the bulk_docs API
33860 · After updating a design document containing a show, an immediate GET
33862 to that same show function may still return results from the previous
33863 definition. This is due to design document caching, which may take a
33864 few seconds to fully evict, or longer (up to ~30s) for a clustered
33865 installation.
33866 Known Issues
33868 All known issues filed against the 2.0 release are contained within the
33869 official CouchDB JIRA instance or CouchDB GitHub Issues.
33870 The following are some highlights of known issues for which fixes did
33872 not land in time for the 2.0.0 release:
33873 · COUCHDB-2980: The replicator (whether invoked via _replicate or a
33875 document stored in the _replicator database) understands two kinds of
33876 source and target:
33877 1. A URL (e.g., https://foo:bar@foo.com/db1), called a “remote”
33879 source or target
33880 2. A database name (e.g., db1), called a “local” source or target.
33882 Whenever the latter type is used, this refers to a local unclustered
33884 database, not a clustered one.
33885 In a future release we hope to support “local” source or target specs
33887 to clustered databases. For now, we recommend always using the URL
33888 format for both source and target specifications.
33889 · COUCHDB-3034: CouchDB will occasionally return 500 errors when multi‐
33891 ple clients attempt to PUT or DELETE the same database concurrently.
33892 · COUCHDB-3119: Adding nodes to a cluster fails if the Erlang node name
33894 is not couchdb (of the form couchdb@hostname.)
33895 · COUCHDB-3050: Occasionally the dev/run script used for development
33897 purposes to start a local 3-node cluster will fail to start one or
33898 more nodes.
33899 · COUCHDB-2817: The compaction daemon will only compact views for
33901 shards that contain the design document.
33902 · COUCHDB-2804: The fast_view optimization is not enabled on the clus‐
33904 tered interface.
33905 · #656: The OAuth 1.0 support is broken and deprecated. It will be
33907 removed in a future version of CouchDB.
33908 Breaking Changes
33910 The following changes in 2.0 represent a significant deviation from
33911 CouchDB 1.x and may alter behaviour of systems designed to work with
33912 older versions of CouchDB:
33913 · #620: POST /dbname no longer returns an ETag response header, in com‐
33915 pliance with RFC 7231, Section 7.2.
33916 1.7.x Branch
33918 · Version 1.7.2
33919 · Version 1.7.1
33921 · Version 1.7.0
33923 Version 1.7.2
33925 Security
33926 · CVE 2018-8007
33927 Version 1.7.1
33929 Bug Fix
33930 · #974: Fix access to /db/_all_docs for database members.
33931 Version 1.7.0
33933 Security
33934 · CVE 2017-12635
33935 · CVE 2017-12636
33937 API Changes
33939 · COUCHDB-1356: Return username on POST /_session.
33940 · COUCHDB-1876: Fix duplicated Content-Type for show/update functions.
33942 · COUCHDB-2310: Implement POST /{db}/_bulk_get.
33944 · COUCHDB-2375: 400 Bad Request returned when invalid revision speci‐
33946 fied.
33947 · COUCHDB-2845: 400 Bad Request returned when revs is not a list.
33949 Build
33951 · COUCHDB-1964: Replace etap test suite with EUnit.
33952 · COUCHDB-2225: Enforce that shared libraries can be built by the sys‐
33954 tem.
33955 · COUCHDB-2761: Support glibc >= 2.20.
33957 · COUCHDB-2747: Support Erlang 18.
33959 · #5b9742c: Support Erlang 19.
33961 · #1545bf4: Remove broken benchmarks.
33963 Database Core
33965 · COUCHDB-2534: Improve checks for db admin/member.
33966 · COUCHDB-2735: Duplicate document _ids created under high edit load.
33968 Documentation
33970 · #c3c9588: Improve documentation of cacert_file ssl option.
33971 · #3266f23: Clarify the purpose of tombstones.
33973 · #75887d9: Improve CouchDB Replication Protocol definition.
33975 · #3b1dc0f: Remove mention of group_level=exact.
33977 · #2a11daa: Remove mention of “Test Suite” in Futon.
33979 · #01c60f1: Clarify type of key, startkey and endkey params.
33981 Futon
33983 · COUCHDB-241: Support document copying.
33984 · COUCHDB-1011: Run replication filtered by document ids from Futon.
33986 · COUCHDB-1275: Unescape database names in Futon recently used list.
33988 · #f18f82a: Update jquery.ui to 1.10.4 with fixes of potential XSS
33990 issues.
33991 HTTP Server
33993 · COUCHDB-2430: Disable Nagle’s algorithm by default.
33994 · COUCHDB-2583: Don’t drop connection by the endpoints which doesn’t
33996 require any payload.
33997 · COUCHDB-2673: Properly escape Location: HTTP header.
33999 · COUCHDB-2677: Wrong Expires header weekday.
34001 · COUCHDB-2783: Bind both to IPv4 and IPv6.
34003 · #f30f3dd: Support for user configurable SSL ciphers.
34005 Query Server
34007 · COUCHDB-1447: Custom response headers from design functions get
34008 merged with default ones.
34009 · #7779c11: Upgrade Coffeescript to version 1.10.
34011 jquery.couch.js
34013 · #f9095e7: Fix document copying.
34014 1.6.x Branch
34016 · Upgrade Notes
34017 · Version 1.6.0
34019 Upgrade Notes
34021 The Proxy Authentication handler was renamed to proxy_authentica‐
34022 tion_handler to follow the *_authentication_handler form of all other
34023 handlers. The old proxy_authentification_handler name is marked as dep‐
34024 recated and will be removed in future releases. It’s strongly recom‐
34025 mended to update httpd/authentication_handlers option with new value in
34026 case if you had used such handler.
34027 Version 1.6.0
34029 · COUCHDB-2200: support Erlang/OTP 17.0 #35e16032
34030 · Fauxton: many improvements in our experimental new user interface,
34032 including switching the code editor from CodeMirror to Ace as well as
34033 better support for various browsers.
34034 · Add the max_count option (config/uuids) to allow rate-limiting the
34036 amount of UUIDs that can be requested from the api/server/uuids han‐
34037 dler in a single request (CVE 2014-2668).
34038 · COUCHDB-1986: increase socket buffer size to improve replication
34040 speed for large documents and attachments, and fix tests on BSD-like
34041 systems. #9a0e561b
34042 · COUCHDB-1953: improve performance of multipart/related requests.
34044 #ce3e89dc
34045 · COUCHDB-2221: verify that authentication-related configuration set‐
34047 tings are well-formed. #dbe769c6
34048 · COUCHDB-1922: fix CORS exposed headers. #4f619833
34050 · Rename proxy_authentification_handler to proxy_authentication_han‐
34052 dler. #c66ac4a8
34053 · COUCHDB-1795: ensure the startup script clears the pid file on termi‐
34055 nation. #818ef4f9
34056 · COUCHDB-1962: replication can now be performed without having write
34058 access to the source database (#1d5fe2aa), the replication checkpoint
34059 interval is now configurable (#0693f98e).
34060 · COUCHDB-2025: add support for SOCKS5 proxies for replication.
34062 #fcd76c9
34063 · COUCHDB-1930: redirect to the correct page after submitting a new
34065 document with a different ID than the one suggested by Futon.
34066 #4906b591
34067 · COUCHDB-1923: add support for attachments and att_encoding_info
34069 options (formerly only available on the documents API) to the view
34070 API. #ca41964b
34071 · COUCHDB-1647: for failed replications originating from a document in
34073 the _replicator database, store the failure reason in the document.
34074 #08cac68b
34075 · A number of improvements for the documentation.
34077 1.5.x Branch
34079 · Version 1.5.1
34080 · Version 1.5.0
34082 WARNING:
34084 Version 1.5.1 contains important security fixes. Previous 1.5.x
34085 releases are not recommended for regular usage.
34086 Version 1.5.1
34088 · Add the max_count option (config/uuids) to allow rate-limiting the
34089 amount of UUIDs that can be requested from the api/server/uuids han‐
34090 dler in a single request (CVE 2014-2668).
34091 Version 1.5.0
34093 · COUCHDB-1781: The official documentation has been overhauled. A lot
34094 of content from other sources have been merged, and the index page
34095 has been rebuilt to make the docs much more accessible. #54813a7
34096 · A new administration UI, codenamed Fauxton, has been included as an
34098 experimental preview. It can be accessed at /_utils/fauxton/. There
34099 are too many improvements here to list them all. We are looking for
34100 feedback from the community on this preview release.
34101 · COUCHDB-1888: Fixed an issue where admin users would be restricted by
34103 the public_fields feature.
34104 · Fixed an issue with the JavaScript CLI test runner. #be76882,
34106 #54813a7
34107 · COUCHDB-1867: An experimental plugin feature has been added. See
34109 src/couch_plugin/README.md for details. We invite the community to
34110 test and report any findings.
34111 · COUCHDB-1894: An experimental Node.js-based query server runtime has
34113 been added. See experimental for details. We invite the community to
34114 test and report any findings.
34115 · COUCHDB-1901: Better retry mechanism for transferring attachments
34117 during replication. #4ca2cec
34118 1.4.x Branch
34120 · Upgrade Notes
34121 · Version 1.4.0
34123 WARNING:
34125 1.4.x Branch is affected by the issue described in cve/2014-2668.
34126 Upgrading to a more recent release is strongly recommended.
34127 Upgrade Notes
34129 We now support Erlang/OTP R16B and R16B01; the minimum required version
34130 is R14B.
34131 User document role values must now be strings. Other types of values
34133 will be refused when saving the user document.
34134 Version 1.4.0
34136 · COUCHDB-1139: it’s possible to apply list functions to _all_docs
34137 view. #54fd258e
34138 · COUCHDB-1632: Ignore epilogues in multipart/related MIME attachments.
34140 #2b4ab67a
34141 · COUCHDB-1634: Reduce PBKDF2 work factor. #f726bc4d
34143 · COUCHDB-1684: Support for server-wide changes feed reporting on cre‐
34145 ation, updates and deletion of databases. #917d8988
34146 · COUCHDB-1772: Prevent invalid JSON output when using all_or_nothing
34148 of bulk API. #dfd39d57
34149 · Add a configurable whitelist of user document properties. #8d7ab8b1
34151 · COUCHDB-1852: Support Last-Event-ID header in EventSource changes
34153 feeds. #dfd2199a
34154 · Allow storing pre-hashed admin passwords via config API. #c98ba561
34156 · Automatic loading of CouchDB plugins. #3fab6bb5
34158 · Much improved documentation, including an expanded description of
34160 validate_doc_update functions (commit:ef9ac469) and a description of
34161 how CouchDB handles JSON number values (#bbd93f77).
34162 · Split up replicator_db tests into multiple independent tests.
34164 1.3.x Branch
34166 · Upgrade Notes
34167 · Version 1.3.1
34169 · Version 1.3.0
34171 WARNING:
34173 1.3.x Branch is affected by the issue described in cve/2014-2668.
34174 Upgrading to a more recent release is strongly recommended.
34175 Upgrade Notes
34177 You can upgrade your existing CouchDB 1.0.x installation to 1.3.0 with‐
34178 out any specific steps or migration. When you run CouchDB, the existing
34179 data and index files will be opened and used as normal.
34180 The first time you run a compaction routine on your database within
34182 1.3.0, the data structure and indexes will be updated to the new ver‐
34183 sion of the CouchDB database format that can only be read by CouchDB
34184 1.3.0 and later. This step is not reversible. Once the data files have
34185 been updated and migrated to the new version the data files will no
34186 longer work with a CouchDB 1.0.x release.
34187 WARNING:
34189 If you want to retain support for opening the data files in CouchDB
34190 1.0.x you must back up your data files before performing the upgrade
34191 and compaction process.
34192 Version 1.3.1
34194 Replicator
34195 · COUCHDB-1788: Tolerate missing source and target fields in _replica‐
34196 tor docs. #869f42e2
34197 Log System
34199 · COUCHDB-1794: Fix bug in WARN level logging from 1.3.0.
34200 · Don’t log about missing .compact files. #06f1a8dc
34202 View Server
34204 · COUCHDB-1792: Fix the -S option to couchjs to increase memory limits.
34205 #cfaa66cd
34206 Miscellaneous
34208 · COUCHDB-1784: Improvements to test suite and VPATH build system.
34209 #01afaa4f
34210 · Improve documentation: better structure, improve language, less
34212 duplication.
34213 Version 1.3.0
34215 Database core
34216 · COUCHDB-1512: Validate bind address before assignment. #09ead8a0
34217 · Restore max_document_size protection. #bf1eb135
34219 Documentation
34221 · COUCHDB-1523: Import CouchBase documentation and convert them into
34222 Sphinx docs
34223 Futon
34225 · COUCHDB-509: Added view request duration to Futon. #2d2c7d1e
34226 · COUCHDB-627: Support all timezones. #b1a049bb
34228 · COUCHDB-1383: Futon view editor won’t allow you to save original view
34230 after saving a revision. #ce48342
34231 · COUCHDB-1470: Futon raises pop-up on attempt to navigate to
34233 missed/deleted document. #5da40eef
34234 · COUCHDB-1473, COUCHDB-1472: Disable buttons for actions that the user
34236 doesn’t have permissions to. #7156254d
34237 HTTP Interface
34239 · COUCHDB-431: Introduce experimental CORS support. #b90e4021
34240 · COUCHDB-764, COUCHDB-514, COUCHDB-430: Fix sending HTTP headers from
34242 _list function, #2a74f88375
34243 · COUCHDB-887: Fix bytes and offset parameters semantic for _log
34245 resource (explanation) #ad700014
34246 · COUCHDB-986: Added Server-Sent Events protocol to db changes API.
34248 See http://www.w3.org/TR/eventsource/ for details. #093d2aa6
34249 · COUCHDB-1026: Database names are encoded with respect of special
34251 characters in the rewriter now. #272d6415
34252 · COUCHDB-1097: Allow OPTIONS request to shows and lists functions.
34254 #9f53704a
34255 · COUCHDB-1210: Files starting with underscore can be attached and
34257 updated now. #05858792
34258 · COUCHDB-1277: Better query parameter support and code clarity:
34260 #7e3c69ba
34261 · Responses to documents created/modified via form data POST to
34263 /db/doc or copied with COPY should now include Location header.
34264 · Form data POST to /db/doc now includes an ETag response header.
34266 · ?batch=ok is now supported for COPY and POST /db/doc updates.
34268 · ?new_edits=false is now supported for more operations.
34270 · COUCHDB-1285: Allow configuration of vendor and modules version in
34272 CouchDB welcome message. #3c24a94d
34273 · COUCHDB-1321: Variables in rewrite rules breaks OAuth authentication.
34275 #c307ba95
34276 · COUCHDB-1337: Use MD5 for attachment ETag header value. #6d912c9f
34278 · COUCHDB-1381: Add jquery.couch support for Windows 8 Metro apps.
34280 #dfc5d37c
34281 · COUCHDB-1441: Limit recursion depth in the URL rewriter. Defaults to
34283 a maximum of 100 invocations but is configurable. #d076976c
34284 · COUCHDB-1442: No longer rewrites the X-CouchDB-Requested-Path during
34286 recursive calls to the rewriter. #56744f2f
34287 · COUCHDB-1501: Changes feed now can take special parameter since=now
34289 to emit changes since current point of time. #3bbb2612
34290 · COUCHDB-1502: Allow users to delete own _users doc. #f0d6f19bc8
34292 · COUCHDB-1511: CouchDB checks roles field for _users database docu‐
34294 ments with more care. #41205000
34295 · COUCHDB-1537: Include user name in show/list ETags. #ac320479
34297 · Send a 202 response for _restart. #b213e16f
34299 · Make password hashing synchronous when using the /_config/admins API.
34301 #08071a80
34302 · Add support to serve single file with CouchDB, #2774531ff2
34304 · Allow any 2xx code to indicate success, #0d50103cfd
34306 · Fix _session for IE7.
34308 · Restore 400 error for empty PUT, #2057b895
34310 · Return X-Couch-Id header if doc is created, #98515bf0b9
34312 · Support auth cookies with : characters, #d9566c831d
34314 Log System
34316 · COUCHDB-1380: Minor fixes for logrotate support.
34317 · Improve file I/O error logging and handling, #4b6475da
34319 · Module Level Logging, #b58f069167
34321 · Log 5xx responses at error level, #e896b0b7
34323 · Log problems opening database at ERROR level except for auto-created
34325 system dbs, #41667642f7
34326 Replicator
34328 · COUCHDB-1248: HTTP 500 error now doesn’t occurs when replicating with
34329 ?doc_ids=null. #bea76dbf
34330 · COUCHDB-1259: Stabilize replication id, #c6252d6d7f
34332 · COUCHDB-1323: Replicator now acts as standalone application.
34334 #f913ca6e
34335 · COUCHDB-1363: Fix rarely occurred, but still race condition in
34337 changes feed if a quick burst of changes happens while replication is
34338 starting the replication can go stale. #573a7bb9
34339 · COUCHDB-1557: Upgrade some code to use BIFs bring good improvements
34341 for replication.
34342 Security
34344 · COUCHDB-1060: Passwords are now hashed using the PBKDF2 algorithm
34345 with a configurable work factor. #7d418134
34346 Source Repository
34348 · The source repository was migrated from SVN to Git.
34349 Storage System
34351 · Fixed unnecessary conflict when deleting and creating a document in
34352 the same batch.
34353 Test Suite
34355 · COUCHDB-1321: Moved the JS test suite to the CLI.
34356 · COUCHDB-1338: Start CouchDB with port=0. While CouchDB might be
34358 already running on the default port 5984, port number 0 let the TCP
34359 stack figure out a free port to run. #127cbe3
34360 · COUCHDB-1339: Use shell trap to catch dying beam processes during
34362 test runs. #2921c78
34363 · COUCHDB-1389: Improved tracebacks printed by the JS CLI tests.
34365 · COUCHDB-1563: Ensures urlPrefix is set in all ajax requests.
34367 #07a6af222
34368 · Fix race condition for test running on faster hardware.
34370 · Improved the reliability of a number of tests.
34372 URL Rewriter & Vhosts
34374 · COUCHDB-1026: Database name is encoded during rewriting (allowing
34375 embedded /’s, etc). #272d6415
34376 UUID Algorithms
34378 · COUCHDB-1373: Added the utc_id algorithm #5ab712a2
34379 Query and View Server
34381 · COUCHDB-111: Improve the errors reported by the JavaScript view
34382 server to provide a more friendly error report when something goes
34383 wrong. #0c619ed
34384 · COUCHDB-410: More graceful error handling for JavaScript vali‐
34386 date_doc_update functions.
34387 · COUCHDB-1372: _stats built-in reduce function no longer produces
34389 error for empty view result.
34390 · COUCHDB-1444: Fix missed_named_view error that occurs on existed
34392 design documents and views. #b59ac98b
34393 · COUCHDB-1445: CouchDB tries no more to delete view file if it
34395 couldn’t open it, even if the error is emfile.
34396 · COUCHDB-1483: Update handlers requires valid doc ids. #72ea7e38
34398 · COUCHDB-1491: Clean up view tables. #c37204b7
34400 · Deprecate E4X support, #cdfdda2314
34402 Windows
34404 · COUCHDB-1482: Use correct linker flag to build snappy_nif.dll on Win‐
34405 dows. #a6eaf9f1
34406 · Allows building cleanly on Windows without cURL, #fb670f5712
34408 1.2.x Branch
34410 · Upgrade Notes
34411 · Version 1.2.2
34413 · Version 1.2.1
34415 · Version 1.2.0
34417 Upgrade Notes
34419 WARNING:
34420 This version drops support for the database format that was intro‐
34421 duced in version 0.9.0. Compact your older databases (that have not
34422 been compacted for a long time) before upgrading, or they will
34423 become inaccessible.
34424 WARNING:
34426 Version 1.2.1 contains important security fixes. Previous 1.2.x
34427 releases are not recommended for regular usage.
34428 Security changes
34430 The interface to the _users and _replicator databases have been changed
34431 so that non-administrator users can see less information:
34432 · In the _users database:
34434 · User documents can now only be read by the respective users, as
34436 well as administrators. Other users cannot read these documents.
34437 · Views can only be defined and queried by administrator users.
34439 · The _changes feed can only be queried by administrator users.
34441 · In the _replicator database:
34443 · Documents now have a forced owner field that corresponds to the
34445 authenticated user that created them.
34446 · Non-owner users will not see confidential information like pass‐
34448 words or OAuth tokens in replication documents; they can still see
34449 the other contents of those documents. Administrators can see
34450 everything.
34451 · Views can only be defined and queried by administrators.
34453 Database Compression
34455 The new optional (but enabled by default) compression of disk files
34456 requires an upgrade of the on-disk format (5 -> 6) which occurs on cre‐
34457 ation for new databases and views, and on compaction for existing
34458 files. This format is not supported in previous releases, so rollback
34459 would require replication to the previous CouchDB release or restoring
34460 from backup.
34461 Compression can be disabled by setting compression = none in your
34463 local.ini [couchdb] section, but the on-disk format will still be
34464 upgraded.
34465 Version 1.2.2
34467 Build System
34468 · Fixed issue in couchdb script where stopped status returns before
34469 process exits.
34470 HTTP Interface
34472 · Reset rewrite counter on new request, avoiding unnecessary request
34473 failures due to bogus rewrite limit reports.
34474 Version 1.2.1
34476 Build System
34477 · Fix couchdb start script.
34478 · Win: fix linker invocations.
34480 Futon
34482 · Disable buttons that aren’t available for the logged-in user.
34483 HTTP Interface
34485 · No longer rewrites the X-CouchDB-Requested-Path during recursive
34486 calls to the rewriter.
34487 · Limit recursion depth in the URL rewriter. Defaults to a maximum of
34489 100 invocations but is configurable.
34490 Security
34492 · Fixed cve/2012-5641
34493 · Fixed cve/2012-5649
34495 · Fixed cve/2012-5650
34497 Replication
34499 · Fix potential timeouts.
34500 View Server
34502 · Change use of signals to avoid broken view groups.
34503 Version 1.2.0
34505 Authentication
34506 · Fix use of OAuth with VHosts and URL rewriting.
34507 · OAuth secrets can now be stored in the users system database as an
34509 alternative to key value pairs in the .ini configuration. By default
34510 this is disabled (secrets are stored in the .ini) but can be enabled
34511 via the .ini configuration key use_users_db in the couch_httpd_oauth
34512 section.
34513 · Documents in the _users database are no longer publicly readable.
34515 · Confidential information in the _replication database is no longer
34517 publicly readable.
34518 · Password hashes are now calculated by CouchDB. Clients are no longer
34520 required to do this manually.
34521 · Cookies used for authentication can be made persistent by enabling
34523 the .ini configuration key allow_persistent_cookies in the
34524 couch_httpd_auth section.
34525 Build System
34527 · cURL is no longer required to build CouchDB as it is only used by the
34528 command line JS test runner. If cURL is available when building
34529 CouchJS you can enable the HTTP bindings by passing -H on the command
34530 line.
34531 · Temporarily made make check pass with R15B. A more thorough fix is in
34533 the works (COUCHDB-1424).
34534 · Fixed –with-js-include and –with-js-lib options.
34536 · Added –with-js-lib-name option.
34538 Futon
34540 · The Status screen (active tasks) now displays two new task status
34541 fields: Started on and Updated on.
34542 · Futon remembers view code every time it is saved, allowing to save an
34544 edit that amounts to a revert.
34545 HTTP Interface
34547 · Added a native JSON parser.
34548 · The _active_tasks API now offers more granular fields. Each task type
34550 is now able to expose different properties.
34551 · Added built-in changes feed filter _view.
34553 · Fixes to the _changes feed heartbeat option which caused heartbeats
34555 to be missed when used with a filter. This caused timeouts of contin‐
34556 uous pull replications with a filter.
34557 · Properly restart the SSL socket on configuration changes.
34559 OAuth
34561 · Updated bundled erlang_oauth library to the latest version.
34562 Replicator
34564 · A new replicator implementation. It offers more performance and con‐
34565 figuration options.
34566 · Passing non-string values to query_params is now a 400 bad request.
34568 This is to reduce the surprise that all parameters are converted to
34569 strings internally.
34570 · Added optional field since_seq to replication objects/documents. It
34572 allows to bootstrap a replication from a specific source sequence
34573 number.
34574 · Simpler replication cancellation. In addition to the current method,
34576 replications can now be canceled by specifying the replication ID
34577 instead of the original replication object/document.
34578 Storage System
34580 · Added optional database and view index file compression (using
34581 Google’s snappy or zlib’s deflate). This feature is enabled by
34582 default, but it can be disabled by adapting local.ini accordingly.
34583 The on-disk format is upgraded on compaction and new DB/view creation
34584 to support this.
34585 · Several performance improvements, most notably regarding database
34587 writes and view indexing.
34588 · Computation of the size of the latest MVCC snapshot data and all its
34590 supporting metadata, both for database and view index files. This
34591 information is exposed as the data_size attribute in the database and
34592 view group information URIs.
34593 · The size of the buffers used for database and view compaction is now
34595 configurable.
34596 · Added support for automatic database and view compaction. This fea‐
34598 ture is disabled by default, but it can be enabled via the .ini con‐
34599 figuration.
34600 · Performance improvements for the built-in changes feed filters
34602 _doc_ids and _design.
34603 View Server
34605 · Add CoffeeScript (http://coffeescript.org/) as a first class view
34606 server language.
34607 · Fixed old index file descriptor leaks after a view cleanup.
34609 · The requested_path property keeps the pre-rewrite path even when no
34611 VHost configuration is matched.
34612 · Fixed incorrect reduce query results when using pagination parame‐
34614 ters.
34615 · Made icu_driver work with Erlang R15B and later.
34617 1.1.x Branch
34619 · Upgrade Notes
34620 · Version 1.1.2
34622 · Version 1.1.1
34624 · Version 1.1.0
34626 Upgrade Notes
34628 WARNING:
34629 Version 1.1.2 contains important security fixes. Previous 1.1.x
34630 releases are not recommended for regular usage.
34631 Version 1.1.2
34633 Build System
34634 · Don’t ln the couchjs install target on Windows
34635 · Remove ICU version dependency on Windows.
34637 · Improve SpiderMonkey version detection.
34639 HTTP Interface
34641 · ETag of attachment changes only when the attachment changes, not the
34642 document.
34643 · Fix retrieval of headers larger than 4k.
34645 · Allow OPTIONS HTTP method for list requests.
34647 · Don’t attempt to encode invalid json.
34649 Log System
34651 · Improvements to log messages for file-related errors.
34652 Replicator
34654 · Fix pull replication of documents with many revisions.
34655 · Fix replication from an HTTP source to an HTTP target.
34657 Security
34659 · Fixed cve/2012-5641
34660 · Fixed cve/2012-5649
34662 · Fixed cve/2012-5650
34664 View Server
34666 · Avoid invalidating view indexes when running out of file descriptors.
34667 Version 1.1.1
34669 · Support SpiderMonkey 1.8.5
34670 · Add configurable maximum to the number of bytes returned by _log.
34672 · Allow CommonJS modules to be an empty string.
34674 · Bump minimum Erlang version to R13B02.
34676 · Do not run deleted validate_doc_update functions.
34678 · ETags for views include current sequence if include_docs=true.
34680 · Fix bug where duplicates can appear in _changes feed.
34682 · Fix bug where update handlers break after conflict resolution.
34684 · Fix bug with _replicator where include “filter” could crash couch.
34686 · Fix crashes when compacting large views.
34688 · Fix file descriptor leak in _log
34690 · Fix missing revisions in _changes?style=all_docs.
34692 · Improve handling of compaction at max_dbs_open limit.
34694 · JSONP responses now send “text/javascript” for Content-Type.
34696 · Link to ICU 4.2 on Windows.
34698 · Permit forward slashes in path to update functions.
34700 · Reap couchjs processes that hit reduce_overflow error.
34702 · Status code can be specified in update handlers.
34704 · Support provides() in show functions.
34706 · _view_cleanup when ddoc has no views now removes all index files.
34708 · max_replication_retry_count now supports “infinity”.
34710 · Fix replication crash when source database has a document with empty
34712 ID.
34713 · Fix deadlock when assigning couchjs processes to serve requests.
34715 · Fixes to the document multipart PUT API.
34717 · Fixes regarding file descriptor leaks for databases with views.
34719 Version 1.1.0
34721 NOTE:
34722 All CHANGES for 1.0.2 and 1.0.3 also apply to 1.1.0.
34723 Externals
34725 · Added OS Process module to manage daemons outside of CouchDB.
34726 · Added HTTP Proxy handler for more scalable externals.
34728 Futon
34730 · Added a “change password”-feature to Futon.
34731 HTTP Interface
34733 · Native SSL support.
34734 · Added support for HTTP range requests for attachments.
34736 · Added built-in filters for _changes: _doc_ids and _design.
34738 · Added configuration option for TCP_NODELAY aka “Nagle”.
34740 · Allow POSTing arguments to _changes.
34742 · Allow keys parameter for GET requests to views.
34744 · Allow wildcards in vhosts definitions.
34746 · More granular ETag support for views.
34748 · More flexible URL rewriter.
34750 · Added support for recognizing “Q values” and media parameters in HTTP
34752 Accept headers.
34753 · Validate doc ids that come from a PUT to a URL.
34755 Replicator
34757 · Added _replicator database to manage replications.
34758 · Fixed issues when an endpoint is a remote database accessible via
34760 SSL.
34761 · Added support for continuous by-doc-IDs replication.
34763 · Fix issue where revision info was omitted when replicating attach‐
34765 ments.
34766 · Integrity of attachment replication is now verified by MD5.
34768 Storage System
34770 · Multiple micro-optimizations when reading data.
34771 URL Rewriter & Vhosts
34773 · Fix for variable substitution
34774 View Server
34776 · Added CommonJS support to map functions.
34777 · Added stale=update_after query option that triggers a view update
34779 after returning a stale=ok response.
34780 · Warn about empty result caused by startkey and endkey limiting.
34782 · Built-in reduce function _sum now accepts lists of integers as input.
34784 · Added view query aliases start_key, end_key, start_key_doc_id and
34786 end_key_doc_id.
34787 1.0.x Branch
34789 · Upgrade Notes
34790 · Version 1.0.4
34792 · Version 1.0.3
34794 · Version 1.0.2
34796 · Version 1.0.1
34798 · Version 1.0.0
34800 Upgrade Notes
34802 Note, to replicate with a 1.0 CouchDB instance you must first upgrade
34803 in-place your current CouchDB to 1.0 or 0.11.1 – backporting so that
34804 0.10.x can replicate to 1.0 wouldn’t be that hard. All that is required
34805 is patching the replicator to use the application/json content type.
34806 · _log and _temp_views are now admin-only resources.
34808 · _bulk_docs now requires a valid Content-Type header of applica‐
34810 tion/json.
34811 · JSONP is disabled by default. An .ini option was added to selectively
34813 enable it.
34814 · The key, startkey and endkey properties of the request object passed
34816 to list and show functions now contain JSON objects representing the
34817 URL encoded string values in the query string. Previously, these
34818 properties contained strings which needed to be converted to JSON
34819 before using.
34820 WARNING:
34822 Version 1.0.4 contains important security fixes. Previous 1.0.x
34823 releases are not recommended for regular usage.
34824 Version 1.0.4
34826 HTTP Interface
34827 · Fix missing revisions in _changes?style=all_docs.
34828 · Fix validation of attachment names.
34830 Log System
34832 · Fix file descriptor leak in _log.
34833 Replicator
34835 · Fix a race condition where replications can go stale
34836 Security
34838 · Fixed cve/2012-5641
34839 · Fixed cve/2012-5649
34841 · Fixed cve/2012-5650
34843 View System
34845 · Avoid invalidating view indexes when running out of file descriptors.
34846 Version 1.0.3
34848 General
34849 · Fixed compatibility issues with Erlang R14B02.
34850 Etap Test Suite
34852 · Etap tests no longer require use of port 5984. They now use a ran‐
34853 domly selected port so they won’t clash with a running CouchDB.
34854 Futon
34856 · Made compatible with jQuery 1.5.x.
34857 HTTP Interface
34859 · Fix bug that allows invalid UTF-8 after valid escapes.
34860 · The query parameter include_docs now honors the parameter conflicts.
34862 This applies to queries against map views, _all_docs and _changes.
34863 · Added support for inclusive_end with reduce views.
34865 Replicator
34867 · Enabled replication over IPv6.
34868 · Fixed for crashes in continuous and filtered changes feeds.
34870 · Fixed error when restarting replications in OTP R14B02.
34872 · Upgrade ibrowse to version 2.2.0.
34874 · Fixed bug when using a filter and a limit of 1.
34876 Security
34878 · Fixed OAuth signature computation in OTP R14B02.
34879 · Handle passwords with : in them.
34881 Storage System
34883 · More performant queries against _changes and _all_docs when using the
34884 include_docs parameter.
34885 Windows
34887 · Windows builds now require ICU >= 4.4.0 and Erlang >= R14B03. See
34888 COUCHDB-1152, and COUCHDB-963 + OTP-9139 for more information.
34889 Version 1.0.2
34891 Futon
34892 · Make test suite work with Safari and Chrome.
34893 · Fixed animated progress spinner.
34895 · Fix raw view document link due to overzealous URI encoding.
34897 · Spell javascript correctly in loadScript(uri).
34899 HTTP Interface
34901 · Allow reduce=false parameter in map-only views.
34902 · Fix parsing of Accept headers.
34904 · Fix for multipart GET APIs when an attachment was created during a
34906 local-local replication. See COUCHDB-1022 for details.
34907 Log System
34909 · Reduce lengthy stack traces.
34910 · Allow logging of native <xml> types.
34912 Replicator
34914 · Updated ibrowse library to 2.1.2 fixing numerous replication issues.
34915 · Make sure that the replicator respects HTTP settings defined in the
34917 config.
34918 · Fix error when the ibrowse connection closes unexpectedly.
34920 · Fix authenticated replication (with HTTP basic auth) of design docu‐
34922 ments with attachments.
34923 · Various fixes to make replication more resilient for edge-cases.
34925 Storage System
34927 · Fix leaking file handles after compacting databases and views.
34928 · Fix databases forgetting their validation function after compaction.
34930 · Fix occasional timeout errors after successfully compacting large
34932 databases.
34933 · Fix occasional error when writing to a database that has just been
34935 compacted.
34936 · Fix occasional timeout errors on systems with slow or heavily loaded
34938 IO.
34939 · Fix for OOME when compactions include documents with many conflicts.
34941 · Fix for missing attachment compression when MIME types included
34943 parameters.
34944 · Preserve purge metadata during compaction to avoid spurious view
34946 rebuilds.
34947 · Fix spurious conflicts introduced when uploading an attachment after
34949 a doc has been in a conflict. See COUCHDB-902 for details.
34950 · Fix for frequently edited documents in multi-master deployments being
34952 duplicated in _changes and _all_docs. See COUCHDB-968 for details on
34953 how to repair.
34954 · Significantly higher read and write throughput against database and
34956 view index files.
34957 View Server
34959 · Don’t trigger view updates when requesting _design/doc/_info.
34960 · Fix for circular references in CommonJS requires.
34962 · Made isArray() function available to functions executed in the query
34964 server.
34965 · Documents are now sealed before being passed to map functions.
34967 · Force view compaction failure when duplicated document data exists.
34969 When this error is seen in the logs users should rebuild their views
34970 from scratch to fix the issue. See COUCHDB-999 for details.
34971 Version 1.0.1
34973 Authentication
34974 ·
34975 Enable basic-auth popup when required to access the server, to pre‐
34977 vent
34978 people from getting locked out.
34979 Build and System Integration
34981 · Included additional source files for distribution.
34982 Futon
34984 · User interface element for querying stale (cached) views.
34985 HTTP Interface
34987 · Expose committed_update_seq for monitoring purposes.
34988 · Show fields saved along with _deleted=true. Allows for auditing of
34990 deletes.
34991 · More robust Accept-header detection.
34993 Replicator
34995 · Added support for replication via an HTTP/HTTPS proxy.
34996 · Fix pull replication of attachments from 0.11 to 1.0.x.
34998 · Make the _changes feed work with non-integer seqnums.
35000 Storage System
35002 · Fix data corruption bug COUCHDB-844. Please see
35003 http://couchdb.apache.org/notice/1.0.1.html for details.
35004 Version 1.0.0
35006 Security
35007 · Added authentication caching, to avoid repeated opening and closing
35008 of the users database for each request requiring authentication.
35009 Storage System
35011 · Small optimization for reordering result lists.
35012 · More efficient header commits.
35014 · Use O_APPEND to save lseeks.
35016 · Faster implementation of pread_iolist(). Further improves performance
35018 on concurrent reads.
35019 View Server
35021 · Faster default view collation.
35022 · Added option to include update_seq in view responses.
35024 0.11.x Branch
35026 · Upgrade Notes
35027 · Version 0.11.2
35029 · Version 0.11.1
35031 · Version 0.11.0
35033 Upgrade Notes
35035 WARNING:
35036 Version 0.11.2 contains important security fixes. Previous 0.11.x
35037 releases are not recommended for regular usage.
35038 Changes Between 0.11.0 and 0.11.1
35040 · _log and _temp_views are now admin-only resources.
35041 · _bulk_docs now requires a valid Content-Type header of applica‐
35043 tion/json.
35044 · JSONP is disabled by default. An .ini option was added to selectively
35046 enable it.
35047 · The key, startkey and endkey properties of the request object passed
35049 to list and show functions now contain JSON objects representing the
35050 URL encoded string values in the query string. Previously, these
35051 properties contained strings which needed to be converted to JSON
35052 before using.
35053 Changes Between 0.10.x and 0.11.0
35055 show, list, update and validation functions
35056 The req argument to show, list, update and validation functions now
35057 contains the member method with the specified HTTP method of the cur‐
35058 rent request. Previously, this member was called verb. method is fol‐
35059 lowing RFC 2616 (HTTP 1.1) closer.
35060 _admins -> _security
35062 The /db/_admins handler has been removed and replaced with a /db/_secu‐
35063 rity object. Any existing _admins will be dropped and need to be added
35064 to the security object again. The reason for this is that the old sys‐
35065 tem made no distinction between names and roles, while the new one
35066 does, so there is no way to automatically upgrade the old admins list.
35067 The security object has 2 special fields, admins and readers, which
35069 contain lists of names and roles which are admins or readers on that
35070 database. Anything else may be stored in other fields on the security
35071 object. The entire object is made available to validation functions.
35072 json2.js
35074 JSON handling in the query server has been upgraded to use json2.js.
35075 This allows us to use faster native JSON serialization when it is
35076 available.
35077 In previous versions, attempts to serialize undefined would throw an
35079 exception, causing the doc that emitted undefined to be dropped from
35080 the view index. The new behavior is to serialize undefined as null.
35081 Applications depending on the old behavior will need to explicitly
35082 check for undefined.
35083 Another change is that E4X’s XML objects will not automatically be
35085 stringified. XML users will need to call my_xml_object.toXMLString() to
35086 return a string value. #8d3b7ab3
35087 WWW-Authenticate
35089 The default configuration has been changed to avoid causing basic-auth
35090 popups which result from sending the WWW-Authenticate header. To enable
35091 basic-auth popups, uncomment the config option httpd/WWW-Authenticate
35092 line in local.ini.
35093 Query server line protocol
35095 The query server line protocol has changed for all functions except
35096 map, reduce, and rereduce. This allows us to cache the entire design
35097 document in the query server process, which results in faster perfor‐
35098 mance for common operations. It also gives more flexibility to query
35099 server implementators and shouldn’t require major changes in the future
35100 when adding new query server features.
35101 UTF8 JSON
35103 JSON request bodies are validated for proper UTF-8 before saving,
35104 instead of waiting to fail on subsequent read requests.
35105 _changes line format
35107 Continuous changes are now newline delimited, instead of having each
35108 line followed by a comma.
35109 Version 0.11.2
35111 Authentication
35112 · User documents can now be deleted by admins or the user.
35113 Futon
35115 · Add some Futon files that were missing from the Makefile.
35116 HTTP Interface
35118 · Better error messages on invalid URL requests.
35119 Replicator
35121 · Fix bug when pushing design docs by non-admins, which was hanging the
35122 replicator for no good reason.
35123 · Fix bug when pulling design documents from a source that requires
35125 basic-auth.
35126 Security
35128 · Avoid potential DOS attack by guarding all creation of atoms.
35129 · Fixed cve/2010-2234
35131 Version 0.11.1
35133 Build and System Integration
35134 · Output of couchdb –help has been improved.
35135 · Fixed compatibility with the Erlang R14 series.
35137 · Fixed warnings on Linux builds.
35139 · Fixed build error when aclocal needs to be called during the build.
35141 · Require ICU 4.3.1.
35143 · Fixed compatibility with Solaris.
35145 Configuration System
35147 · Fixed timeout with large .ini files.
35148 Futon
35150 · Use “expando links” for over-long document values in Futon.
35151 · Added continuous replication option.
35153 · Added option to replicating test results anonymously to a community
35155 CouchDB instance.
35156 · Allow creation and deletion of config entries.
35158 · Fixed display issues with doc ids that have escaped characters.
35160 · Fixed various UI issues.
35162 HTTP Interface
35164 · Mask passwords in active tasks and logging.
35165 · Update mochijson2 to allow output of BigNums not in float form.
35167 · Added support for X-HTTP-METHOD-OVERRIDE.
35169 · Better error message for database names.
35171 · Disable jsonp by default.
35173 · Accept gzip encoded standalone attachments.
35175 · Made max_concurrent_connections configurable.
35177 · Made changes API more robust.
35179 · Send newly generated document rev to callers of an update function.
35181 JavaScript Clients
35183 · Added tests for couch.js and jquery.couch.js
35184 · Added changes handler to jquery.couch.js.
35186 · Added cache busting to jquery.couch.js if the user agent is msie.
35188 · Added support for multi-document-fetch (via _all_docs) to
35190 jquery.couch.js.
35191 · Added attachment versioning to jquery.couch.js.
35193 · Added option to control ensure_full_commit to jquery.couch.js.
35195 · Added list functionality to jquery.couch.js.
35197 · Fixed issues where bulkSave() wasn’t sending a POST body.
35199 Log System
35201 · Log HEAD requests as HEAD, not GET.
35202 · Keep massive JSON blobs out of the error log.
35204 · Fixed a timeout issue.
35206 Replication System
35208 · Refactored various internal APIs related to attachment streaming.
35209 · Fixed hanging replication.
35211 · Fixed keepalive issue.
35213 Security
35215 · Added authentication redirect URL to log in clients.
35216 · Fixed query parameter encoding issue in oauth.js.
35218 · Made authentication timeout configurable.
35220 · Temporary views are now admin-only resources.
35222 Storage System
35224 · Don’t require a revpos for attachment stubs.
35225 · Added checking to ensure when a revpos is sent with an attachment
35227 stub, it’s correct.
35228 · Make file deletions async to avoid pauses during compaction and db
35230 deletion.
35231 · Fixed for wrong offset when writing headers and converting them to
35233 blocks, only triggered when header is larger than 4k.
35234 · Preserve _revs_limit and instance_start_time after compaction.
35236 Test Suite
35238 · Made the test suite overall more reliable.
35239 View Server
35241 · Provide a UUID to update functions (and all other functions) that
35242 they can use to create new docs.
35243 · Upgrade CommonJS modules support to 1.1.1.
35245 · Fixed erlang filter funs and normalize filter fun API.
35247 · Fixed hang in view shutdown.
35249 URL Rewriter & Vhosts
35251 · Allow more complex keys in rewriter.
35252 · Allow global rewrites so system defaults are available in vhosts.
35254 · Allow isolation of databases with vhosts.
35256 · Fix issue with passing variables to query parameters.
35258 Version 0.11.0
35260 Build and System Integration
35261 · Updated and improved source documentation.
35262 · Fixed distribution preparation for building on Mac OS X.
35264 · Added support for building a Windows installer as part of ‘make
35266 dist’.
35267 · Bug fix for building couch.app’s module list.
35269 · ETap tests are now run during make distcheck. This included a number
35271 of updates to the build system to properly support VPATH builds.
35272 · Gavin McDonald set up a build-bot instance. More info can be found at
35274 http://ci.apache.org/buildbot.html
35275 Futon
35277 · Added a button for view compaction.
35278 · JSON strings are now displayed as-is in the document view, without
35280 the escaping of new-lines and quotes. That dramatically improves
35281 readability of multi-line strings.
35282 · Same goes for editing of JSON string values. When a change to a field
35284 value is submitted, and the value is not valid JSON it is assumed to
35285 be a string. This improves editing of multi-line strings a lot.
35286 · Hitting tab in textareas no longer moves focus to the next form
35288 field, but simply inserts a tab character at the current caret posi‐
35289 tion.
35290 · Fixed some font declarations.
35292 HTTP Interface
35294 · Provide Content-MD5 header support for attachments.
35295 · Added URL Rewriter handler.
35297 · Added virtual host handling.
35299 Replication
35301 · Added option to implicitly create replication target databases.
35302 · Avoid leaking file descriptors on automatic replication restarts.
35304 · Added option to replicate a list of documents by id.
35306 · Allow continuous replication to be cancelled.
35308 Runtime Statistics
35310 · Statistics are now calculated for a moving window instead of
35311 non-overlapping timeframes.
35312 · Fixed a problem with statistics timers and system sleep.
35314 · Moved statistic names to a term file in the priv directory.
35316 Security
35318 · Fixed CVE-2010-0009: Apache CouchDB Timing Attack Vulnerability.
35319 · Added default cookie-authentication and users database.
35321 · Added Futon user interface for user signup and login.
35323 · Added per-database reader access control lists.
35325 · Added per-database security object for configuration data in valida‐
35327 tion functions.
35328 · Added proxy authentication handler
35330 Storage System
35332 · Adds batching of multiple updating requests, to improve throughput
35333 with many writers. Removed the now redundant couch_batch_save module.
35334 · Adds configurable compression of attachments.
35336 View Server
35338 · Added optional ‘raw’ binary collation for faster view builds where
35339 Unicode collation is not important.
35340 · Improved view index build time by reducing ICU collation callouts.
35342 · Improved view information objects.
35344 · Bug fix for partial updates during view builds.
35346 · Move query server to a design-doc based protocol.
35348 · Use json2.js for JSON serialization for compatiblity with native
35350 JSON.
35351 · Major refactoring of couchjs to lay the groundwork for disabling cURL
35353 support. The new HTTP interaction acts like a synchronous XHR. Exam‐
35354 ple usage of the new system is in the JavaScript CLI test runner.
35355 0.10.x Branch
35357 · Upgrade Notes
35358 · Version 0.10.2
35360 · Version 0.10.1
35362 · Version 0.10.0
35364 Upgrade Notes
35366 WARNING:
35367 Version 0.10.2 contains important security fixes. Previous 0.10.x
35368 releases are not recommended for regular usage.
35369 Modular Configuration Directories
35371 CouchDB now loads configuration from the following places (glob(7) syn‐
35372 tax) in order:
35373 · PREFIX/default.ini
35375 · PREFIX/default.d/*
35377 · PREFIX/local.ini
35379 · PREFIX/local.d/*
35381 The configuration options for couchdb script have changed to:
35383 -a FILE add configuration FILE to chain
35385 -A DIR add configuration DIR to chain
35386 -n reset configuration file chain (including system default)
35387 -c print configuration file chain and exit
35388 Show and List API change
35390 Show and List functions must have a new structure in 0.10. See
35391 Formatting_with_Show_and_List for details.
35392 Stricter enforcing of reduciness in reduce-functions
35394 Reduce functions are now required to reduce the number of values for a
35395 key.
35396 View query reduce parameter strictness
35398 CouchDB now considers the parameter reduce=false to be an error for
35399 queries of map-only views, and responds with status code 400.
35400 Version 0.10.2
35402 Build and System Integration
35403 · Fixed distribution preparation for building on Mac OS X.
35404 Security
35406 · Fixed cve/2010-0009
35407 Replicator
35409 · Avoid leaking file descriptors on automatic replication restarts.
35410 Version 0.10.1
35412 Build and System Integration
35413 · Test suite now works with the distcheck target.
35414 Replicator
35416 · Stability enhancements regarding redirects, timeouts, OAuth.
35417 Query Server
35419 · Avoid process leaks
35420 · Allow list and view to span languages
35422 Stats
35424 · Eliminate new process flood on system wake
35425 Version 0.10.0
35427 Build and System Integration
35428 · Changed couchdb script configuration options.
35429 · Added default.d and local.d configuration directories to load
35431 sequence.
35432 HTTP Interface
35434 · Added optional cookie-based authentication handler.
35435 · Added optional two-legged OAuth authentication handler.
35437 Storage Format
35439 · Add move headers with checksums to the end of database files for
35440 extra robust storage and faster storage.
35441 View Server
35443 · Added native Erlang views for high-performance applications.
35444 0.9.x Branch
35446 · Upgrade Notes
35447 · Version 0.9.2
35449 · Version 0.9.1
35451 · Version 0.9.0
35453 Upgrade Notes
35455 Response to Bulk Creation/Updates
35456 The response to a bulk creation / update now looks like this
35457 [
35459 {"id": "0", "rev": "3682408536"},
35460 {"id": "1", "rev": "3206753266"},
35461 {"id": "2", "error": "conflict", "reason": "Document update conflict."}
35462 ]
35463 Database File Format
35465 The database file format has changed. CouchDB itself does yet not pro‐
35466 vide any tools for migrating your data. In the meantime, you can use
35467 third-party scripts to deal with the migration, such as the dump/load
35468 tools that come with the development version (trunk) of couchdb-python.
35469 Renamed “count” to “limit”
35471 The view query API has been changed: count has become limit. This is a
35472 better description of what the parameter does, and should be a simple
35473 update in any client code.
35474 Moved View URLs
35476 The view URLs have been moved to design document resources. This means
35477 that paths that used to be like:
35478 http://hostname:5984/mydb/_view/designname/viewname?limit=10
35480 will now look like:
35482 http://hostname:5984/mydb/_design/designname/_view/viewname?limit=10.
35484 See the REST, Hypermedia, and CouchApps thread on dev for details.
35486 Attachments
35488 Names of attachments are no longer allowed to start with an underscore.
35489 Error Codes
35491 Some refinements have been made to error handling. CouchDB will send
35492 400 instead of 500 on invalid query parameters. Most notably, document
35493 update conflicts now respond with 409 Conflict instead of 412 Precondi‐
35494 tion Failed. The error code for when attempting to create a database
35495 that already exists is now 412 instead of 409.
35496 ini file format
35498 CouchDB 0.9 changes sections and configuration variable names in con‐
35499 figuration files. Old .ini files won’t work. Also note that CouchDB now
35500 ships with two .ini files where 0.8 used couch.ini there are now
35501 default.ini and local.ini. default.ini contains CouchDB’s standard
35502 configuration values. local.ini is meant for local changes. local.ini
35503 is not overwritten on CouchDB updates, so your edits are safe. In addi‐
35504 tion, the new runtime configuration system persists changes to the con‐
35505 figuration in local.ini.
35506 Version 0.9.2
35508 Build and System Integration
35509 · Remove branch callbacks to allow building couchjs against newer ver‐
35510 sions of Spidermonkey.
35511 Replication
35513 · Fix replication with 0.10 servers initiated by an 0.9 server (‐
35514 COUCHDB-559).
35515 Version 0.9.1
35517 Build and System Integration
35518 · PID file directory is now created by the SysV/BSD daemon scripts.
35519 · Fixed the environment variables shown by the configure script.
35521 · Fixed the build instructions shown by the configure script.
35523 · Updated ownership and permission advice in README for better secu‐
35525 rity.
35526 Configuration and stats system
35528 · Corrected missing configuration file error message.
35529 · Fixed incorrect recording of request time.
35531 Database Core
35533 · Document validation for underscore prefixed variables.
35534 · Made attachment storage less sparse.
35536 · Fixed problems when a database with delayed commits pending is con‐
35538 sidered idle, and subject to losing changes when shutdown. (‐
35539 COUCHDB-334)
35540 External Handlers
35542 · Fix POST requests.
35543 Futon
35545 · Redirect when loading a deleted view URI from the cookie.
35546 HTTP Interface
35548 · Attachment requests respect the “rev” query-string parameter.
35549 JavaScript View Server
35551 · Useful JavaScript Error messages.
35552 Replication
35554 · Added support for Unicode characters transmitted as UTF-16 surrogate
35555 pairs.
35556 · URL-encode attachment names when necessary.
35558 · Pull specific revisions of an attachment, instead of just the latest
35560 one.
35561 · Work around a rare chunk-merging problem in ibrowse.
35563 · Work with documents containing Unicode characters outside the Basic
35565 Multilingual Plane.
35566 Version 0.9.0
35568 Build and System Integration
35569 · The couchdb script now supports system chainable configuration files.
35570 · The Mac OS X daemon script now redirects STDOUT and STDERR like
35572 SysV/BSD.
35573 · The build and system integration have been improved for portability.
35575 · Added COUCHDB_OPTIONS to etc/default/couchdb file.
35577 · Remove COUCHDB_INI_FILE and COUCHDB_PID_FILE from etc/default/couchdb
35579 file.
35580 · Updated configure.ac to manually link libm for portability.
35582 · Updated configure.ac to extended default library paths.
35584 · Removed inets configuration files.
35586 · Added command line test runner.
35588 · Created dev target for make.
35590 Configuration and stats system
35592 · Separate default and local configuration files.
35593 · HTTP interface for configuration changes.
35595 · Statistics framework with HTTP query API.
35597 Database Core
35599 · Faster B-tree implementation.
35600 · Changed internal JSON term format.
35602 · Improvements to Erlang VM interactions under heavy load.
35604 · User context and administrator role.
35606 · Update validations with design document validation functions.
35608 · Document purge functionality.
35610 · Ref-counting for database file handles.
35612 Design Document Resource Paths
35614 · Added httpd_design_handlers config section.
35615 · Moved _view to httpd_design_handlers.
35617 · Added ability to render documents as non-JSON content-types with
35619 _show and _list functions, which are also httpd_design_handlers.
35620 Futon Utility Client
35622 · Added pagination to the database listing page.
35623 · Implemented attachment uploading from the document page.
35625 · Added page that shows the current configuration, and allows modifica‐
35627 tion of option values.
35628 · Added a JSON “source view” for document display.
35630 · JSON data in view rows is now syntax highlighted.
35632 · Removed the use of an iframe for better integration with browser his‐
35634 tory and bookmarking.
35635 · Full database listing in the sidebar has been replaced by a short
35637 list of recent databases.
35638 · The view editor now allows selection of the view language if there is
35640 more than one configured.
35641 · Added links to go to the raw view or document URI.
35643 · Added status page to display currently running tasks in CouchDB.
35645 · JavaScript test suite split into multiple files.
35647 · Pagination for reduce views.
35649 HTTP Interface
35651 · Added client side UUIDs for idempotent document creation
35652 · HTTP COPY for documents
35654 · Streaming of chunked attachment PUTs to disk
35656 · Remove negative count feature
35658 · Add include_docs option for view queries
35660 · Add multi-key view post for views
35662 · Query parameter validation
35664 · Use stale=ok to request potentially cached view index
35666 · External query handler module for full-text or other indexers.
35668 · Etags for attachments, views, shows and lists
35670 · Show and list functions for rendering documents and views as devel‐
35672 oper controlled content-types.
35673 · Attachment names may use slashes to allow uploading of nested direc‐
35675 tories (useful for static web hosting).
35676 · Option for a view to run over design documents.
35678 · Added newline to JSON responses. Closes bike-shed.
35680 Replication
35682 · Using ibrowse.
35683 · Checkpoint replications so failures are less expensive.
35685 · Automatically retry of failed replications.
35687 · Stream attachments in pull-replication.
35689 0.8.x Branch
35691 · Version 0.8.1-incubating
35692 · Version 0.8.0-incubating
35694 Version 0.8.1-incubating
35696 Build and System Integration
35697 · The couchdb script no longer uses awk for configuration checks as
35698 this was causing portability problems.
35699 · Updated sudo example in README to use the -i option, this fixes prob‐
35701 lems when invoking from a directory the couchdb user cannot access.
35702 Database Core
35704 · Fix for replication problems where the write queues can get backed up
35705 if the writes aren’t happening fast enough to keep up with the reads.
35706 For a large replication, this can exhaust memory and crash, or slow
35707 down the machine dramatically. The fix keeps only one document in the
35708 write queue at a time.
35709 · Fix for databases sometimes incorrectly reporting that they contain 0
35711 documents after compaction.
35712 · CouchDB now uses ibrowse instead of inets for its internal HTTP
35714 client implementation. This means better replication stability.
35715 Futon
35717 · The view selector dropdown should now work in Opera and Internet
35718 Explorer even when it includes optgroups for design documents. (‐
35719 COUCHDB-81)
35720 JavaScript View Server
35722 · Sealing of documents has been disabled due to an incompatibility with
35723 SpiderMonkey 1.9.
35724 · Improve error handling for undefined values emitted by map functions.
35726 (COUCHDB-83)
35727 HTTP Interface
35729 · Fix for chunked responses where chunks were always being split into
35730 multiple TCP packets, which caused problems with the test suite under
35731 Safari, and in some other cases.
35732 · Fix for an invalid JSON response body being returned for some kinds
35734 of views. (COUCHDB-84)
35735 · Fix for connections not getting closed after rejecting a chunked
35737 request. (COUCHDB-55)
35738 · CouchDB can now be bound to IPv6 addresses.
35740 · The HTTP Server header now contains the versions of CouchDB and
35742 Erlang.
35743 Version 0.8.0-incubating
35745 Build and System Integration
35746 · CouchDB can automatically respawn following a server crash.
35747 · Database server no longer refuses to start with a stale PID file.
35749 · System logrotate configuration provided.
35751 · Improved handling of ICU shared libraries.
35753 · The couchdb script now automatically enables SMP support in Erlang.
35755 · The couchdb and couchjs scripts have been improved for portability.
35757 · The build and system integration have been improved for portability.
35759 Database Core
35761 · The view engine has been completely decoupled from the storage
35762 engine. Index data is now stored in separate files, and the format of
35763 the main database file has changed.
35764 · Databases can now be compacted to reclaim space used for deleted doc‐
35766 uments and old document revisions.
35767 · Support for incremental map/reduce views has been added.
35769 · To support map/reduce, the structure of design documents has changed.
35771 View values are now JSON objects containing at least a map member,
35772 and optionally a reduce member.
35773 · View servers are now identified by name (for example javascript)
35775 instead of by media type.
35776 · Automatically generated document IDs are now based on proper UUID
35778 generation using the crypto module.
35779 · The field content-type in the JSON representation of attachments has
35781 been renamed to content_type (underscore).
35782 Futon
35784 · When adding a field to a document, Futon now just adds a field with
35785 an autogenerated name instead of prompting for the name with a dia‐
35786 log. The name is automatically put into edit mode so that it can be
35787 changed immediately.
35788 · Fields are now sorted alphabetically by name when a document is dis‐
35790 played.
35791 · Futon can be used to create and update permanent views.
35793 · The maximum number of rows to display per page on the database page
35795 can now be adjusted.
35796 · Futon now uses the XMLHTTPRequest API asynchronously to communicate
35798 with the CouchDB HTTP server, so that most operations no longer block
35799 the browser.
35800 · View results sorting can now be switched between ascending and
35802 descending by clicking on the Key column header.
35803 · Fixed a bug where documents that contained a @ character could not be
35805 viewed. (COUCHDB-12)
35806 · The database page now provides a Compact button to trigger database
35808 compaction. (COUCHDB-38)
35809 · Fixed portential double encoding of document IDs and other URI seg‐
35811 ments in many instances. (COUCHDB-39)
35812 · Improved display of attachments.
35814 · The JavaScript Shell has been removed due to unresolved licensing
35816 issues.
35817 JavaScript View Server
35819 · SpiderMonkey is no longer included with CouchDB, but rather treated
35820 as a normal external dependency. A simple C program (_couchjs) is
35821 provided that links against an existing SpiderMonkey installation and
35822 uses the interpreter embedding API.
35823 · View functions using the default JavaScript view server can now do
35825 logging using the global log(message) function. Log messages are
35826 directed into the CouchDB log at INFO level. (COUCHDB-59)
35827 · The global map(key, value) function made available to view code has
35829 been renamed to emit(key, value).
35830 · Fixed handling of exceptions raised by view functions.
35832 HTTP Interface
35834 · CouchDB now uses MochiWeb instead of inets for the HTTP server imple‐
35835 mentation. Among other things, this means that the extra configura‐
35836 tion files needed for inets (such as couch_httpd.conf) are no longer
35837 used.
35838 · The HTTP interface now completely supports the HEAD method. (‐
35840 COUCHDB-3)
35841 · Improved compliance of Etag handling with the HTTP specification. (‐
35843 COUCHDB-13)
35844 · Etags are no longer included in responses to document GET requests
35846 that include query string parameters causing the JSON response to
35847 change without the revision or the URI having changed.
35848 · The bulk document update API has changed slightly on both the request
35850 and the response side. In addition, bulk updates are now atomic.
35851 · CouchDB now uses TCP_NODELAY to fix performance problems with persis‐
35853 tent connections on some platforms due to nagling.
35854 · Including a ?descending=false query string parameter in requests to
35856 views no longer raises an error.
35857 · Requests to unknown top-level reserved URLs (anything with a leading
35859 underscore) now return a unknown_private_path error instead of the
35860 confusing illegal_database_name.
35861 · The Temporary view handling now expects a JSON request body, where
35863 the JSON is an object with at least a map member, and optional reduce
35864 and language members.
35865 · Temporary views no longer determine the view server based on the Con‐
35867 tent-Type header of the POST request, but rather by looking for a
35868 language member in the JSON body of the request.
35869 · The status code of responses to DELETE requests is now 200 to reflect
35871 that that the deletion is performed synchronously.
35872
35874 CVE-2010-0009: Apache CouchDB Timing Attack Vulnerability
35875 Date 31.03.2010
35876 Affected
35878 Apache CouchDB 0.8.0 to 0.10.1
35879 Severity
35881 Important
35882 Vendor The Apache Software Foundation
35884 Description
35886 Apache CouchDB versions prior to version 0.11.0 are vulnerable to tim‐
35887 ing attacks, also known as side-channel information leakage, due to
35888 using simple break-on-inequality string comparisons when verifying
35889 hashes and passwords.
35890 Mitigation
35892 All users should upgrade to CouchDB 0.11.0. Upgrades from the 0.10.x
35893 series should be seamless. Users on earlier versions should consult
35894 with upgrade notes.
35895 Example
35897 A canonical description of the attack can be found in
35898 http://codahale.com/a-lesson-in-timing-attacks/
35899 Credit
35901 This issue was discovered by Jason Davies of the Apache CouchDB devel‐
35902 opment team.
35903 CVE-2010-2234: Apache CouchDB Cross Site Request Forgery Attack
35905 Date 21.02.2010
35906 Affected
35908 Apache CouchDB 0.8.0 to 0.11.1
35909 Severity
35911 Important
35912 Vendor The Apache Software Foundation
35914 Description
35916 Apache CouchDB versions prior to version 0.11.1 are vulnerable to Cross
35917 Site Request Forgery (CSRF) attacks.
35918 Mitigation
35920 All users should upgrade to CouchDB 0.11.2 or 1.0.1.
35921 Upgrades from the 0.11.x and 0.10.x series should be seamless.
35923 Users on earlier versions should consult with upgrade notes.
35925 Example
35927 A malicious website can POST arbitrary JavaScript code to well known
35928 CouchDB installation URLs (like http://localhost:5984/) and make the
35929 browser execute the injected JavaScript in the security context of
35930 CouchDB’s admin interface Futon.
35931 Unrelated, but in addition the JSONP API has been turned off by default
35933 to avoid potential information leakage.
35934 Credit
35936 This CSRF issue was discovered by a source that wishes to stay anony‐
35937 mous.
35938 CVE-2010-3854: Apache CouchDB Cross Site Scripting Issue
35940 Date 28.01.2011
35941 Affected
35943 Apache CouchDB 0.8.0 to 1.0.1
35944 Severity
35946 Important
35947 Vendor The Apache Software Foundation
35949 Description
35951 Apache CouchDB versions prior to version 1.0.2 are vulnerable to Cross
35952 Site Scripting (XSS) attacks.
35953 Mitigation
35955 All users should upgrade to CouchDB 1.0.2.
35956 Upgrades from the 0.11.x and 0.10.x series should be seamless.
35958 Users on earlier versions should consult with upgrade notes.
35960 Example
35962 Due to inadequate validation of request parameters and cookie data in
35963 Futon, CouchDB’s web-based administration UI, a malicious site can exe‐
35964 cute arbitrary code in the context of a user’s browsing session.
35965 Credit
35967 This XSS issue was discovered by a source that wishes to stay anony‐
35968 mous.
35969 CVE-2012-5641: Information disclosure via unescaped backslashes in URLs on
35971 Windows
35972 Date 14.01.2013
35973 Affected
35975 All Windows-based releases of Apache CouchDB, up to and includ‐
35976 ing 1.0.3, 1.1.1, and 1.2.0 are vulnerable.
35977 Severity
35979 Moderate
35980 Vendor The Apache Software Foundation
35982 Description
35984 A specially crafted request could be used to access content directly
35985 that would otherwise be protected by inbuilt CouchDB security mecha‐
35986 nisms. This request could retrieve in binary form any CouchDB database,
35987 including the _users or _replication databases, or any other file that
35988 the user account used to run CouchDB might have read access to on the
35989 local filesystem. This exploit is due to a vulnerability in the
35990 included MochiWeb HTTP library.
35991 Mitigation
35993 Upgrade to a supported CouchDB release that includes this fix, such as:
35994 · 1.0.4
35996 · 1.1.2
35998 · 1.2.1
36000 · 1.3.x
36002 All listed releases have included a specific fix for the MochiWeb com‐
36004 ponent.
36005 Work-Around
36007 Users may simply exclude any file-based web serving components directly
36008 within their configuration file, typically in local.ini. On a default
36009 CouchDB installation, this requires amending the httpd_global_han‐
36010 dlers/favicon.ico and httpd_global_handlers/_utils lines within
36011 httpd_global_handlers:
36012 [httpd_global_handlers]
36014 favicon.ico = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36015 _utils = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36016 If additional handlers have been added, such as to support Adobe’s
36018 Flash crossdomain.xml files, these would also need to be excluded.
36019 Acknowledgement
36021 The issue was found and reported by Sriram Melkote to the upstream
36022 MochiWeb project.
36023 References
36025 · https://github.com/melkote/mochiweb/commit/ac2bf
36026 CVE-2012-5649: JSONP arbitrary code execution with Adobe Flash
36028 Date 14.01.2013
36029 Affected
36031 Releases up to and including 1.0.3, 1.1.1, and 1.2.0 are vulner‐
36032 able, if administrators have enabled JSONP.
36033 Severity
36035 Moderate
36036 Vendor The Apache Software Foundation
36038 Description
36040 A hand-crafted JSONP callback and response can be used to run arbitrary
36041 code inside client-side browsers via Adobe Flash.
36042 Mitigation
36044 Upgrade to a supported CouchDB release that includes this fix, such as:
36045 · 1.0.4
36047 · 1.1.2
36049 · 1.2.1
36051 · 1.3.x
36053 All listed releases have included a specific fix.
36055 Work-Around
36057 Disable JSONP or don’t enable it since it’s disabled by default.
36058 CVE-2012-5650: DOM based Cross-Site Scripting via Futon UI
36060 Date 14.01.2013
36061 Affected
36063 Apache CouchDB releases up to and including 1.0.3, 1.1.1, and
36064 1.2.0 are vulnerable.
36065 Severity
36067 Moderate
36068 Vendor The Apache Software Foundation
36070 Description
36072 Query parameters passed into the browser-based test suite are not sani‐
36073 tised, and can be used to load external resources. An attacker may exe‐
36074 cute JavaScript code in the browser, using the context of the remote
36075 user.
36076 Mitigation
36078 Upgrade to a supported CouchDB release that includes this fix, such as:
36079 · 1.0.4
36081 · 1.1.2
36083 · 1.2.1
36085 · 1.3.x
36087 All listed releases have included a specific fix.
36089 Work-Around
36091 Disable the Futon user interface completely, by adapting local.ini and
36092 restarting CouchDB:
36093 [httpd_global_handlers]
36095 _utils = {couch_httpd_misc_handlers, handle_welcome_req, <<"Forbidden">>}
36096 Or by removing the UI test suite components:
36098 · share/www/verify_install.html
36100 · share/www/couch_tests.html
36102 · share/www/custom_test.html
36104 Acknowledgement
36106 This vulnerability was discovered & reported to the Apache Software
36107 Foundation by Frederik Braun.
36108 CVE-2014-2668: DoS (CPU and memory consumption) via the count parameter to
36110 /_uuids
36111 Date 26.03.2014
36112 Affected
36114 Apache CouchDB releases up to and including 1.3.1, 1.4.0, and
36115 1.5.0 are vulnerable.
36116 Severity
36118 Moderate
36119 Vendor The Apache Software Foundation
36121 Description
36123 The api/server/uuids resource’s count query parameter is able to take
36124 unreasonable huge numeric value which leads to exhaustion of server
36125 resources (CPU and memory) and to DoS as the result.
36126 Mitigation
36128 Upgrade to a supported CouchDB release that includes this fix, such as:
36129 · 1.5.1
36131 · 1.6.0
36133 All listed releases have included a specific fix to
36135 Work-Around
36137 Disable the api/server/uuids handler completely, by adapting local.ini
36138 and restarting CouchDB:
36139 [httpd_global_handlers]
36141 _uuids =
36142 CVE-2017-12635: Apache CouchDB Remote Privilege Escalation
36144 Date 14.11.2017
36145 Affected
36147 All Versions of Apache CouchDB
36148 Severity
36150 Critical
36151 Vendor The Apache Software Foundation
36153 Description
36155 Due to differences in CouchDB’s Erlang-based JSON parser and
36156 JavaScript-based JSON parser, it is possible to submit _users documents
36157 with duplicate keys for roles used for access control within the data‐
36158 base, including the special case _admin role, that denotes administra‐
36159 tive users. In combination with CVE-2017-12636 (Remote Code Execution),
36160 this can be used to give non-admin users access to arbitrary shell com‐
36161 mands on the server as the database system user.
36162 Mitigation
36164 All users should upgrade to CouchDB 1.7.1 or 2.1.1.
36165 Upgrades from previous 1.x and 2.x versions in the same series should
36167 be seamless.
36168 Users on earlier versions, or users upgrading from 1.x to 2.x should
36170 consult with upgrade notes.
36171 Example
36173 The JSON parser differences result in behaviour that if two roles keys
36174 are available in the JSON, the second one will be used for authorising
36175 the document write, but the first roles key is used for subsequent
36176 authorisation for the newly created user. By design, users can not
36177 assign themselves roles. The vulnerability allows non-admin users to
36178 give themselves admin privileges.
36179 We addressed this issue by updating the way CouchDB parses JSON in
36181 Erlang, mimicking the JavaScript behaviour of picking the last key, if
36182 duplicates exist.
36183 Credit
36185 This issue was discovered by Max Justicz.
36186 CVE-2017-12636: Apache CouchDB Remote Code Execution
36188 Date 14.11.2017
36189 Affected
36191 All Versions of Apache CouchDB
36192 Severity
36194 Critical
36195 Vendor The Apache Software Foundation
36197 Description
36199 CouchDB administrative users can configure the database server via
36200 HTTP(S). Some of the configuration options include paths for operating
36201 system-level binaries that are subsequently launched by CouchDB. This
36202 allows a CouchDB admin user to execute arbitrary shell commands as the
36203 CouchDB user, including downloading and executing scripts from the pub‐
36204 lic internet.
36205 Mitigation
36207 All users should upgrade to CouchDB 1.7.1 or 2.1.1.
36208 Upgrades from previous 1.x and 2.x versions in the same series should
36210 be seamless.
36211 Users on earlier versions, or users upgrading from 1.x to 2.x should
36213 consult with upgrade notes.
36214 Credit
36216 This issue was discovered by Joan Touzet of the CouchDB Security team
36217 during the investigation of CVE-2017-12635.
36218 CVE-2018-11769: Apache CouchDB Remote Code Execution
36220 Date 08.08.2018
36221 Affected
36223 Apache CouchDB 1.x and ≤2.1.2
36224 Severity
36226 Low
36227 Vendor The Apache Software Foundation
36229 Description
36231 CouchDB administrative users can configure the database server via
36232 HTTP(S). Due to insufficient validation of administrator-supplied con‐
36233 figuration settings via the HTTP API, it is possible for a CouchDB
36234 administrator user to escalate their privileges to that of the operat‐
36235 ing system’s user under which CouchDB runs, by bypassing the blacklist
36236 of configuration settings that are not allowed to be modified via the
36237 HTTP API.
36238 This privilege escalation effectively allows a CouchDB admin user to
36240 gain arbitrary remote code execution, bypassing mitigations for
36241 CVE-2017-12636 and CVE-2018-8007.
36242 Mitigation
36244 All users should upgrade to CouchDB 2.2.0.
36245 Upgrades from previous 2.x versions in the same series should be seam‐
36247 less.
36248 Users still on CouchDB 1.x should be advised that the Apache CouchDB
36250 team no longer support 1.x.
36251 In-place mitigation (on any 1.x release, or 2.x prior to 2.2.0) is pos‐
36253 sible by removing the _config route from the default.ini file, as fol‐
36254 lows:
36255 [httpd_global_handlers]
36257 ;_config = {couch_httpd_misc_handlers, handle_config_req}
36258 or by blocking access to the /_config (1.x) or /_node/*/_config routes
36260 at a reverse proxy in front of the service.
36261 CVE-2018-17188: Apache CouchDB Remote Privilege Escalations
36263 Date 17.12.2018
36264 Affected
36266 All Versions of Apache CouchDB
36267 Severity
36269 Medium
36270 Vendor The Apache Software Foundation
36272 Description
36274 Prior to CouchDB version 2.3.0, CouchDB allowed for runtime-configura‐
36275 tion of key components of the database. In some cases, this lead to
36276 vulnerabilities where CouchDB admin users could access the underlying
36277 operating system as the CouchDB user. Together with other vulnerabili‐
36278 ties, it allowed full system entry for unauthenticated users.
36279 These vulnerabilities were fixed and disclosed in the following CVE
36281 reports:
36282 · CVE-2018-11769: Apache CouchDB Remote Code Execution
36284 · CVE-2018-8007: Apache CouchDB Remote Code Execution
36286 · CVE-2017-12636: Apache CouchDB Remote Code Execution
36288 · CVE-2017-12635: Apache CouchDB Remote Privilege Escalation
36290 Rather than waiting for new vulnerabilities to be discovered, and fix‐
36292 ing them as they come up, the CouchDB development team decided to make
36293 changes to avoid this entire class of vulnerabilities.
36294 With CouchDB version 2.3.0, CouchDB no longer can configure key compo‐
36296 nents at runtime. While some flexibility is needed for speciality con‐
36297 figurations of CouchDB, the configuration was changed from being avail‐
36298 able at runtime to start-up time. And as such now requires shell access
36299 to the CouchDB server.
36300 This closes all future paths for vulnerabilities of this type.
36302 Mitigation
36304 All users should upgrade to CouchDB 2.3.0.
36305 Upgrades from previous 2.x versions in the same series should be seam‐
36307 less.
36308 Users on earlier versions should consult with upgrade notes.
36310 Credit
36312 This issue was discovered by the Apple Information Security team.
36313 CVE-2018-8007: Apache CouchDB Remote Code Execution
36315 Date 30.04.2018
36316 Affected
36318 All Versions of Apache CouchDB
36319 Severity
36321 Low
36322 Vendor The Apache Software Foundation
36324 Description
36326 CouchDB administrative users can configure the database server via
36327 HTTP(S). Due to insufficient validation of administrator-supplied con‐
36328 figuration settings via the HTTP API, it is possible for a CouchDB
36329 administrator user to escalate their privileges to that of the operat‐
36330 ing system’s user that CouchDB runs under, by bypassing the backlist of
36331 configuration settings that are not allowed to be modified via the HTTP
36332 API.
36333 This privilege escalation effectively allows a CouchDB admin user to
36335 gain arbitrary remote code execution, bypassing CVE-2017-12636
36336 Mitigation
36338 All users should upgrade to CouchDB 1.7.2 or 2.1.2.
36339 Upgrades from previous 1.x and 2.x versions in the same series should
36341 be seamless.
36342 Users on earlier versions, or users upgrading from 1.x to 2.x should
36344 consult with upgrade notes.
36345 Credit
36347 This issue was discovered by Francesco Oddo of MDSec Labs.
36348
36350 The Apache Software Foundation takes a very active stance in eliminat‐
36351 ing security problems and denial of service attacks against Apache
36352 CouchDB.
36353 We strongly encourage folks to report such problems to our private
36355 security mailing list first, before disclosing them in a public forum.
36356 Please note that the security mailing list should only be used for
36358 reporting undisclosed security vulnerabilities in Apache CouchDB and
36359 managing the process of fixing such vulnerabilities. We cannot accept
36360 regular bug reports or other queries at this address. All mail sent to
36361 this address that does not relate to an undisclosed security problem in
36362 the Apache CouchDB source code will be ignored.
36363 If you need to report a bug that isn’t an undisclosed security vulnera‐
36365 bility, please use the bug reporting page.
36366 Questions about:
36368 · How to configure CouchDB securely
36370 · If a vulnerability applies to your particular application
36372 · Obtaining further information on a published vulnerability
36374 · Availability of patches and/or new releases
36376 should be address to the users mailing list. Please see the mailing
36378 lists page for details of how to subscribe.
36379 The private security mailing address is: security@couchdb.apache.org
36381 Please read how the Apache Software Foundation handles security reports
36383 to know what to expect.
36384 Note that all networked servers are subject to denial of service
36386 attacks, and we cannot promise magic workarounds to generic problems
36387 (such as a client streaming lots of data to your server, or re-request‐
36388 ing the same URL repeatedly). In general our philosophy is to avoid any
36389 attacks which can cause the server to consume resources in a non-linear
36390 relationship to the size of inputs.
36391
36393 License
36394 Apache License
36395 Version 2.0, January 2004
36396 http://www.apache.org/licenses/
36397 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
36399 1. Definitions.
36401 "License" shall mean the terms and conditions for use, reproduction,
36403 and distribution as defined by Sections 1 through 9 of this document.
36404 "Licensor" shall mean the copyright owner or entity authorized by
36406 the copyright owner that is granting the License.
36407 "Legal Entity" shall mean the union of the acting entity and all
36409 other entities that control, are controlled by, or are under common
36410 control with that entity. For the purposes of this definition,
36411 "control" means (i) the power, direct or indirect, to cause the
36412 direction or management of such entity, whether by contract or
36413 otherwise, or (ii) ownership of fifty percent (50%) or more of the
36414 outstanding shares, or (iii) beneficial ownership of such entity.
36415 "You" (or "Your") shall mean an individual or Legal Entity
36417 exercising permissions granted by this License.
36418 "Source" form shall mean the preferred form for making modifications,
36420 including but not limited to software source code, documentation
36421 source, and configuration files.
36422 "Object" form shall mean any form resulting from mechanical
36424 transformation or translation of a Source form, including but
36425 not limited to compiled object code, generated documentation,
36426 and conversions to other media types.
36427 "Work" shall mean the work of authorship, whether in Source or
36429 Object form, made available under the License, as indicated by a
36430 copyright notice that is included in or attached to the work
36431 (an example is provided in the Appendix below).
36432 "Derivative Works" shall mean any work, whether in Source or Object
36434 form, that is based on (or derived from) the Work and for which the
36435 editorial revisions, annotations, elaborations, or other modifications
36436 represent, as a whole, an original work of authorship. For the purposes
36437 of this License, Derivative Works shall not include works that remain
36438 separable from, or merely link (or bind by name) to the interfaces of,
36439 the Work and Derivative Works thereof.
36440 "Contribution" shall mean any work of authorship, including
36442 the original version of the Work and any modifications or additions
36443 to that Work or Derivative Works thereof, that is intentionally
36444 submitted to Licensor for inclusion in the Work by the copyright owner
36445 or by an individual or Legal Entity authorized to submit on behalf of
36446 the copyright owner. For the purposes of this definition, "submitted"
36447 means any form of electronic, verbal, or written communication sent
36448 to the Licensor or its representatives, including but not limited to
36449 communication on electronic mailing lists, source code control systems,
36450 and issue tracking systems that are managed by, or on behalf of, the
36451 Licensor for the purpose of discussing and improving the Work, but
36452 excluding communication that is conspicuously marked or otherwise
36453 designated in writing by the copyright owner as "Not a Contribution."
36454 "Contributor" shall mean Licensor and any individual or Legal Entity
36456 on behalf of whom a Contribution has been received by Licensor and
36457 subsequently incorporated within the Work.
36458 2. Grant of Copyright License. Subject to the terms and conditions of
36460 this License, each Contributor hereby grants to You a perpetual,
36461 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
36462 copyright license to reproduce, prepare Derivative Works of,
36463 publicly display, publicly perform, sublicense, and distribute the
36464 Work and such Derivative Works in Source or Object form.
36465 3. Grant of Patent License. Subject to the terms and conditions of
36467 this License, each Contributor hereby grants to You a perpetual,
36468 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
36469 (except as stated in this section) patent license to make, have made,
36470 use, offer to sell, sell, import, and otherwise transfer the Work,
36471 where such license applies only to those patent claims licensable
36472 by such Contributor that are necessarily infringed by their
36473 Contribution(s) alone or by combination of their Contribution(s)
36474 with the Work to which such Contribution(s) was submitted. If You
36475 institute patent litigation against any entity (including a
36476 cross-claim or counterclaim in a lawsuit) alleging that the Work
36477 or a Contribution incorporated within the Work constitutes direct
36478 or contributory patent infringement, then any patent licenses
36479 granted to You under this License for that Work shall terminate
36480 as of the date such litigation is filed.
36481 4. Redistribution. You may reproduce and distribute copies of the
36483 Work or Derivative Works thereof in any medium, with or without
36484 modifications, and in Source or Object form, provided that You
36485 meet the following conditions:
36486 (a) You must give any other recipients of the Work or
36488 Derivative Works a copy of this License; and
36489 (b) You must cause any modified files to carry prominent notices
36491 stating that You changed the files; and
36492 (c) You must retain, in the Source form of any Derivative Works
36494 that You distribute, all copyright, patent, trademark, and
36495 attribution notices from the Source form of the Work,
36496 excluding those notices that do not pertain to any part of
36497 the Derivative Works; and
36498 (d) If the Work includes a "NOTICE" text file as part of its
36500 distribution, then any Derivative Works that You distribute must
36501 include a readable copy of the attribution notices contained
36502 within such NOTICE file, excluding those notices that do not
36503 pertain to any part of the Derivative Works, in at least one
36504 of the following places: within a NOTICE text file distributed
36505 as part of the Derivative Works; within the Source form or
36506 documentation, if provided along with the Derivative Works; or,
36507 within a display generated by the Derivative Works, if and
36508 wherever such third-party notices normally appear. The contents
36509 of the NOTICE file are for informational purposes only and
36510 do not modify the License. You may add Your own attribution
36511 notices within Derivative Works that You distribute, alongside
36512 or as an addendum to the NOTICE text from the Work, provided
36513 that such additional attribution notices cannot be construed
36514 as modifying the License.
36515 You may add Your own copyright statement to Your modifications and
36517 may provide additional or different license terms and conditions
36518 for use, reproduction, or distribution of Your modifications, or
36519 for any such Derivative Works as a whole, provided Your use,
36520 reproduction, and distribution of the Work otherwise complies with
36521 the conditions stated in this License.
36522 5. Submission of Contributions. Unless You explicitly state otherwise,
36524 any Contribution intentionally submitted for inclusion in the Work
36525 by You to the Licensor shall be under the terms and conditions of
36526 this License, without any additional terms or conditions.
36527 Notwithstanding the above, nothing herein shall supersede or modify
36528 the terms of any separate license agreement you may have executed
36529 with Licensor regarding such Contributions.
36530 6. Trademarks. This License does not grant permission to use the trade
36532 names, trademarks, service marks, or product names of the Licensor,
36533 except as required for reasonable and customary use in describing the
36534 origin of the Work and reproducing the content of the NOTICE file.
36535 7. Disclaimer of Warranty. Unless required by applicable law or
36537 agreed to in writing, Licensor provides the Work (and each
36538 Contributor provides its Contributions) on an "AS IS" BASIS,
36539 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
36540 implied, including, without limitation, any warranties or conditions
36541 of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
36542 PARTICULAR PURPOSE. You are solely responsible for determining the
36543 appropriateness of using or redistributing the Work and assume any
36544 risks associated with Your exercise of permissions under this License.
36545 8. Limitation of Liability. In no event and under no legal theory,
36547 whether in tort (including negligence), contract, or otherwise,
36548 unless required by applicable law (such as deliberate and grossly
36549 negligent acts) or agreed to in writing, shall any Contributor be
36550 liable to You for damages, including any direct, indirect, special,
36551 incidental, or consequential damages of any character arising as a
36552 result of this License or out of the use or inability to use the
36553 Work (including but not limited to damages for loss of goodwill,
36554 work stoppage, computer failure or malfunction, or any and all
36555 other commercial damages or losses), even if such Contributor
36556 has been advised of the possibility of such damages.
36557 9. Accepting Warranty or Additional Liability. While redistributing
36559 the Work or Derivative Works thereof, You may choose to offer,
36560 and charge a fee for, acceptance of support, warranty, indemnity,
36561 or other liability obligations and/or rights consistent with this
36562 License. However, in accepting such obligations, You may act only
36563 on Your own behalf and on Your sole responsibility, not on behalf
36564 of any other Contributor, and only if You agree to indemnify,
36565 defend, and hold each Contributor harmless for any liability
36566 incurred by, or claims asserted against, such Contributor by reason
36567 of your accepting any such warranty or additional liability.
36568 END OF TERMS AND CONDITIONS
36570 APPENDIX: How to apply the Apache License to your work.
36572 To apply the Apache License to your work, attach the following
36574 boilerplate notice, with the fields enclosed by brackets "[]"
36575 replaced with your own identifying information. (Don't include
36576 the brackets!) The text should be enclosed in the appropriate
36577 comment syntax for the file format. We also recommend that a
36578 file or class name and description of purpose be included on the
36579 same "printed page" as the copyright notice for easier
36580 identification within third-party archives.
36581 Copyright [yyyy] [name of copyright owner]
36583 Licensed under the Apache License, Version 2.0 (the "License");
36585 you may not use this file except in compliance with the License.
36586 You may obtain a copy of the License at
36587 http://www.apache.org/licenses/LICENSE-2.0
36589 Unless required by applicable law or agreed to in writing, software
36591 distributed under the License is distributed on an "AS IS" BASIS,
36592 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
36593 See the License for the specific language governing permissions and
36594 limitations under the License.
36595
36598 The documentation lives in its own source tree. We’ll start by forking
36599 and cloning the CouchDB documentation GitHub mirror. That will allow us
36600 to send the contribution to CouchDB with a pull request.
36601 If you don’t have a GitHub account yet, it is a good time to get one,
36603 they are free. If you don’t want to use GitHub, there are alternate
36604 ways to contributing back, that we’ll cover next time.
36605 Go to https://github.com/apache/couchdb-documentation and click the
36607 “fork” button in the top right. This will create a fork of CouchDB in
36608 your GitHub account. If your account is username, your fork lives at
36609 https://github.com/username/couchdb-documentation. In the header, it
36610 tells me my “GitHub Clone URL”. We need to copy that and start a termi‐
36611 nal:
36612 $ git clone https://github.com/username/couchdb-documentation.git
36614 $ cd couchdb-documentation
36615 $ subl .
36616 I’m opening the whole CouchDB documentation source tree in my favourite
36618 editor. It gives me the usual directory listing:
36619 ebin/
36621 ext/
36622 .git/
36623 .gitignore
36624 images/
36625 LICENSE
36626 make.bat
36627 Makefile
36628 NOTICE
36629 rebar.config
36630 src/
36631 static/
36632 templates/
36633 themes/
36634 .travis.yml
36635 The documentation sources live in src, you can safely ignore all the
36637 other files and directories.
36638 First we should determine where we want to document this inside the
36640 documentation. We can look through http://docs.couchdb.org/en/latest/
36641 for inspiration. The JSON Structure Reference looks like a fine place
36642 to write this up.
36643 The current state includes mostly tables describing the JSON structure
36645 (after all, that’s the title of this chapter), but some prose about the
36646 number representation can’t hurt. For future reference, since the topic
36647 in the thread includes views and different encoding in views (as
36648 opposed to the storage engine), we should remember to make a note in
36649 the views documentation as well, but we’ll leave this for later.
36650 Let’s try and find the source file that builds the file
36652 http://docs.couchdb.org/en/latest/json-structure.html – we are in luck,
36653 under share/doc/src we find the file json-structure.rst. That looks
36654 promising. .rst stands for ReStructured Text (see
36655 http://thomas-cokelaer.info/tutorials/sphinx/rest_syntax.html for a
36656 markup reference), which is an ASCII format for writing documents, doc‐
36657 umentation in this case. Let’s have a look and open it.
36658 We see ASCII tables with some additional formatting, all looking like
36660 the final HTML. So far so easy. For now, let’s just add to the bottom
36661 of this. We can worry about organising this better later.
36662 We start by adding a new headline:
36664 Number Handling
36666 ===============
36667 Now we paste in the rest of the main email of the thread. It is mostly
36669 text, but it includes some code listings. Let’s mark them up. We’ll
36670 turn:
36671 ejson:encode(ejson:decode(<<"1.1">>)).
36673 <<"1.1000000000000000888">>
36674 Into:
36676 .. code-block:: erlang
36678 ejson:encode(ejson:decode(<<"1.1">>)).
36680 <<"1.1000000000000000888">>
36681 And we follow along with the other code samples. We turn:
36683 Spidermonkey
36685 $ js -h 2>&1 | head -n 1
36687 JavaScript-C 1.8.5 2011-03-31
36688 $ js
36689 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
36690 "1.0123456789012346"
36691 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
36692 js> JSON.stringify(JSON.parse(f))
36693 "1.0123456789012346"
36694 into:
36696 Spidermonkey::
36698 $ js -h 2>&1 | head -n 1
36700 JavaScript-C 1.8.5 2011-03-31
36701 $ js
36702 js> JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
36703 "1.0123456789012346"
36704 js> var f = JSON.stringify(JSON.parse("1.01234567890123456789012345678901234567890"))
36705 js> JSON.stringify(JSON.parse(f))
36706 "1.0123456789012346"
36707 And then follow all the other ones.
36709 I cleaned up the text a little but to make it sound more like a docu‐
36711 mentation entry as opposed to a post on a mailing list.
36712 The next step would be to validate that we got all the markup right.
36714 I’ll leave this for later. For now we’ll contribute our change back to
36715 CouchDB.
36716 First, we commit our changes:
36718 $ > git commit -am 'document number encoding'
36720 [master a84b2cf] document number encoding
36721 1 file changed, 199 insertions(+)
36722 Then we push the commit to our CouchDB fork:
36724 $ git push origin master
36726 Next, we go back to our GitHub page
36728 https://github.com/username/couchdb-documentation and click the “Pull
36729 Request” button. Fill in the description with something useful and hit
36730 the “Send Pull Request” button.
36731 And we’re done!
36733 Style Guidelines for this Documentation
36735 When you make a change to the documentation, you should make sure that
36736 you follow the style. Look through some files and you will see that the
36737 style is quite straightforward. If you do not know if your formating is
36738 in compliance with the style, ask yourself the following question:
36739 Is it needed for correct syntax?
36741 If the answer is No. then it is probably not.
36743 These guidelines strive be simple, without contradictions and excep‐
36745 tions. The best style is the one that is followed because it seems to
36746 be the natural way of doing it.
36747 The guidelines
36749 The guidelines are in descending priority.
36750 1. Syntax
36752 · Correct syntax is always more important than style. This includes
36754 configuration files, HTML responses, etc.
36755 2. Encoding
36757 · All files are UTF-8.
36759 3. Line ending
36761 · All lines end with \n.
36763 · No trailing whitespace.
36765 4. Line length
36767 · The maximum line length is 80 characters.
36769 5. Links
36771 · All internal links are relative.
36773 6. Indentation
36775 · 4 spaces.
36777 7. Titles
36779 · The highest level titles in a file is over and underlined with =.
36781 · Lower level titles are underlined with the following characters in
36783 descending order:
36784 = - ^ * + # ` : . " ~ _
36786 · Over and underline match the title length.
36788 8. Empty lines
36790 · No empty line at the end of the file.
36792 · Lists may separated each item with an empty line.
36794
36796 2020, Apache Software Foundation. CouchDB® is a registered trademark of
36797 the Apache Software Foundation
367983.0 Feb 19, 2020 APACHECOUCHDB(1)