MongoDB: The ugly things

Here is a list of all the ugly things that I have found so far regarding MongoDB:

Writes that return errors to the client due to wtimeout may in fact succeed (link)

Regarding the option wtimeout:

This option specifies a time limit, in milliseconds, for the write concern. wtimeout is only applicable for w values greater than 1.
wtimeout causes write operations to return with an error after the specified limit, even if the required write concern will eventually succeed. When these write operations return, MongoDB does not undo successful data modifications performed before the write concern exceeded the wtimeout time limit.
If you do not specify the wtimeout option and the level of write concern is unachievable, the write operation will block indefinitely. Specifying a wtimeout value of 0 is equivalent to a write concern without the wtimeout option.

WTF?

Maximum document size is 16MB (link)

The maximum BSON document size is 16 megabytes.

A client may think that a write was successful but it wasn't!

Even with (w=1 or w='majority') and (j=1), it is possible to wind up rolling back a committed write to the primary on failover (link) and the client may think that the write was successful but it wasn't!

Examples

• The network TCP connection between the application and the server was reset after the server received a write but before a response could be sent.
• The MongoDB server terminates between receiving the write and responding to it.
• The network fails between the time of the write and the time the client receives a response to the write. 

w='majority' doesn't resolve the problem because the parameter j doesn't affect the replicas (link):

Requiring journaled write concern in a replica set only requires a journal commit of the write operation to the primary of the set regardless of the level of replica acknowledged write concern.

 You can find more information on the official MongoDB docs and a relative StackOverflow post.

A client doesn't get a response after a write and thinks that the write didn't happen but happened!

Usually the MongoDB client (e.g. Java driver) tackles the following problems transparently to the developer:

• insert isn't acknowledged --> re-execute it (provided the _id was set by the client and is the same on both executions)
• update isn't acknowledged --> treat it as deletion and insertion 
• deletion isn't acknowledged --> idempotent; re-execute it
• find isn't acknowledged --> idempotent; re-execute it

During failover writes cannot occur (i.e. they fail)

When the primary node fails, another is elected from its replica set. During the election the client cannot write. This is because in MongoDB only the primary node can accept writes.

Therefore you may have to guard your code and try to re-execute the failed write.
By the way can the MongoDB driver do anything about it?