-
Notifications
You must be signed in to change notification settings - Fork 181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
monstache - problem reading from MongoDB secondaries. #724
Comments
interesting finding. I was just debugging a similar issue about missing some documents, since I've had used switched to
Thanks for pointing that out, probably saved me a few hours of debugging :-) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We have observed a problem when our Monstache instance uses MongoDB read preference "
secondary
". When we try to insert a large number of documents into MongoDB in a short time, such as 5,000 documents, not all of them are replicated to Elasticsearch. Approximately 4,950 to 4,970 are replicated. However, when we switch Monstache back to read preference "primary
", everything is replicated correctly.All documents are also correctly replicated if we use the connection string to MongoDB with only the secondary MongoDB node name and the parameter
directConnection=true
. However, in this case, Monstache cannot insert metadata to the MongoDB database.Monstache uses a MongoDB view as the replication source in MongoDB. Here is our configuration:
Could the problem be that Monstache sees the document ID in the oplog, takes that document ID, and sends a query to one of the secondaries, e.g.,
db.contact-view.find({"id":"xyz"})
. However, the document is not yet replicated despite being in the oplog, so it gets zero documents as a result of the query ?we use MongoDB
v6.0
, Monstachev6.7.10
The text was updated successfully, but these errors were encountered: