linux - MongoError: topology was destroyed -


i have node-based rest api has started receiving "topology destroyed" errors mongodb.

i'll try give information possible.

architecture:

the system running on 2 node (0.12.7) servers behind nginx reverse-proxy connected single mongodb server via private network. node processes running 1 per server under pm2.

the mongodb server latest version (3.0) , hosts 2 wiredtiger databases. new system being rolled out existing application both growing rapidly 1 database @ 260k small documents, growing @ rate of ~300 documents minute, , other @ 1400 large(r) ones.

the second database has single collection sparse unique key index.

the node servers connect mongodb using mongojs (a thin wrapper around mongodb-core. i've checked source , don't think it's issue) 5 sockets per database set automatically reconnect on failure.

the issue:

in milliseconds before or after (1 - 3 ms) latter of 2 or more update requests trigger duplicate key error on second database, mongodb logs 5 connections database (and not other one) closing on both servers (at first, they're second apart). no error besides expected duplicate key returned, if connections close before error logged.

the next request happens after receives "topology destroyed" error. i've implemented quick fix restarts node server once error received since mongodb not automatically reconnect.

other information , assumptions:

  • since both servers close connections @ same time, though they're unaware of each other , hosted separately, assume issue database , not application.
  • only single database's connections terminated, issue not network related.
  • almost operations simple , take less 5ms, analytics flushes of 500 documents (to larger, unaffected, database) take 200ms. since these times below socket timeouts, assume that's not issue.
  • the servers handling 40 - 90 requests per minute right now, they're never idle long enough connection timeout.
  • the database logging no unusual errors.

does have idea why happening? or happening?


Comments

Popular posts from this blog

dns - How To Use Custom Nameserver On Free Cloudflare? -

python - Pygame screen.blit not working -

c# - Web API response xml language -