{"resource":{"author":{"id":"h2so9H8jPMmgUKGghoNl","name":"Yomesh Gupta","username":"yomeshgupta"},"content":{"link":"https://devtools.tech/speed-up-mongodb-search-queries/","difficulty":2,"domain":3,"type":1,"isInternal":true,"body":"This is blog post is about a problem I faced regarding Search Queries in MongoDB. This is my first blog post so I wanted to start with a simple topic. Feel free to share the feedback.\n\nSo, NoSQL databases are growing in popularity and are used by everyone, from big tech giants to independent developers. NoSQL databases support varied storage approaches such as key-value, document-based and many more. These databases allow developers to store documents that contain various data types. For example, consider a movie database where every document contains a movie `id` and movie `title`.\n\n```js\n{\n  \"id\": 1,\n  \"title\": \"Dawn of Justice\"\n}\n```\n\nTo find the movie with the title `Dawn of Justice`, we can simply run\n\n```js\ndb.movies.find({ title: \"Dawn of Justice\" });\n```\n\nThis command will give us an exact match for the provided string as it will translate to (in SQL terms)\n\n```sql\nselect *\nfrom movies\nwhere title = \"Dawn of Justice\"\n```\n\nHowever, our users would not input search queries like this. They could enter something like `_dawn of justice_, _Dawn of justice_, _DAWN OF JUSTICE_` or any non-exact search query.\n\n![User Search Queries](https://ik.imagekit.io/devtoolstech/speed-up-mongodb-search-queries/user-search-queries-compressed_vZMZL-QYO.gif?ik-sdk-version=javascript-1.4.3&updatedAt=1642921746119)\n\nHere, `_find()_` would not work. In such cases, we can use `Regular Expressions (regex)` which provides a way to match strings against a pattern and MongoDB comes with a built-in regex engine. We can use a query like below where `i` makes the regex case-insensitive.\n\n```js\ndb.movies.find({ title: /Dawn of Justice/i });\n```\n\nWe can leave it to that but as soon as your database size grows, the fetch time of this query will increase and will consume a lot of CPU time.\n\n![Movie Search](https://ik.imagekit.io/devtoolstech/speed-up-mongodb-search-queries/movie-search-min_QtJrK1EdwnW.gif?ik-sdk-version=javascript-1.4.3&updatedAt=1642921745263)\n\nAs you can see in the above demonstration, due to the large size of the database, it takes more than 20 seconds to return the response for a simple search and users will not wait that long. So, what should we do now?\n\n## Text Indexes to the Rescue\n\nStarting from version 2.4, MongoDB supports text indexes to search inside string content. A text index will tokenize and stem the content of the field as in it will break the string into individual words or tokens, and will further reduce them to their stems so that variants of the same word will match. For example, \"talk\" matching \"talks\", \"talked\" and \"talking\" as \"talk\" is a stem of all three. We can create a text index via\n\n```js\ndb.movies.createIndex({ title: \"text\" });\n```\n\nMongoDB provides a `$text` operator which we can use to query data:\n\n```js\ndb.movies.find({ title: { $text: { $search: \"Dawn of Justice\" } });\n```\n\nThis will improve the speed but we will still not get the desired results.\n\n## Power of Text Search with Regular Expressions\n\nWe can use the `$and` operator provided by MongoDB to join Text Search with Regular Expressions. We apply logically `and` to first find a board resultset using Text Search and filter it out using Regular Expressions. If the first condition (text search) fails then regex won't work.\n\nWe can do this by creating a query like\n\n```js\ndb.movies.find({\n  $and: [\n    {\n      $text: {\n        $search: \"Dawn of Justice\",\n      },\n    },\n    {\n      title: {\n        $regex: /Dawn of Justice/i,\n      },\n    },\n  ],\n});\n```\n\nHere, we find a superset of all movies with the title containing the text we are searching for and then filter the resultset with a regex query. If text search yields no result then regex won't execute at all.\n\nThis combination will reduce fetch time and CPU load than using regex queries alone.\n\n![Final Output](https://ik.imagekit.io/devtoolstech/speed-up-mongodb-search-queries/final-output-min_e1nhSB4j4.gif?ik-sdk-version=javascript-1.4.3&updatedAt=1642921744752)\n\nThis is one of the ways to improve your database performance. I learned these optimizations while working on a side project, which is an ML-powered recommendation system. You can check it out here: [re.yomeshgupta.com](https://re.yomeshgupta.com)\n","languages":[],"editorConfig":{}},"stats":{"views":45083,"used":0,"likes":0},"description":"","published":true,"isActive":true,"tags":["node.js","mongodb","backend","database","db","tools","performance","mongo queries"],"slug":"speed-up-mongodb-search-queries---rid---1KlpHxujFKhN2HVJ3AQk","isPremium":false,"categories":[],"requires":[],"_id":"5f1dd537cbec5f7ffc0c2fa8","title":"Speed up MongoDB Search Queries","resourceId":"1KlpHxujFKhN2HVJ3AQk","createdAt":1595790647919,"modifiedAt":1643031137803},"currentUser":null,"isOwner":false,"recommendations":{"questions":[{"_id":"627e4bf2d3473370ac35949a","content":{"languages":["javascript","typescript"],"difficulty":1},"tags":["javascript","beginner","frontend fundamentals","js objects","json","prototype","inheritance","object create","basic js","js interview questions","devtools tech","frontend masters"],"slug":"how-to-create-an-object-without-prototype-or-javascript-interview-question-or-beginner---qid---0zvLanNBGjJKuWUeXLKe","title":"How to create an object without prototype? | JavaScript Interview Question | Beginner","questionId":"0zvLanNBGjJKuWUeXLKe"},{"_id":"642540c26577005718b79302","content":{"languages":["javascript","typescript"],"difficulty":1},"tags":["javascript","frontend","code","ui","ux","devtools tech","lodash","programming","coding","codedamn","egghead","frontend masters","aws","amazon","amazon frontend interview",""],"slug":"how-to-implement-deep-filter-functionality-or-aws-frontend-interview-question-or-javascript-or-problem-solving-or-amazon---qid---aGvxJwFTMSJBpOXqKXfw","title":"How to implement Deep Filter Functionality? | AWS Frontend Interview Question | JavaScript | Problem Solving | Amazon","questionId":"aGvxJwFTMSJBpOXqKXfw"},{"_id":"637f360e4956ba15eb978a02","content":{"languages":["react","html"],"difficulty":1},"tags":["javascript","frontend","code","ui","programming","web animations","css","apple","tv commericals","crazy ones","frontend coding challenge","javascript projects","web projects","web fundamentals","sass","react","vanilla js","text to audio"],"slug":"build-a-text-to-audio-animated-mini-project-or-frontend-coding-challenge-or-javascript-or-css-or-web-animations---qid---HEpSJMMktHpZZSTOva36","title":"Build a Text to Audio Animated Mini Project | Frontend Coding Challenge | JavaScript | CSS | Web Animations","questionId":"HEpSJMMktHpZZSTOva36"},{"_id":"61f8ef79e12bce537863b4cf","content":{"languages":["javascript","typescript"],"difficulty":4},"tags":["javascript","walmart","frontend interview question","devtools","ui","advanced javascript"],"slug":"implement-throttling-in-javascript-or-walmart-frontend-interview-question---qid---mLrwAyug9CgrVGbs6OYF","title":"Implement Throttling in JavaScript | Walmart Frontend Interview Question","questionId":"mLrwAyug9CgrVGbs6OYF"},{"_id":"635172709efd8c366a2e80f4","content":{"languages":["javascript","typescript"],"difficulty":1},"tags":["","javascript","arrays","javascript polyfills","interview question","frontend","coding","devtools tech","programming","codedamn","frontend masters","egghead","razorpay","hotstar","tata1mg","swiggy","zomato","array","array findLastIndex"],"slug":"how-to-implement-array-prototype-findlastindex-javascript-interview-question-or-problem-solving-or-javascript-polyfills---qid---irZ7BX4HZ8iVyTJ7YDrv","title":"How to implement Array.prototype.findLastIndex? JavaScript Interview Question | Problem Solving | JavaScript Polyfills","questionId":"irZ7BX4HZ8iVyTJ7YDrv"}],"resources":[{"_id":"5f1dedf2cbec5f7ffc0c2fb3","content":{"difficulty":2,"domain":2,"type":1,"isInternal":true,"languages":[]},"tags":["node.js","javascript","temporal dead zone","js tdz","es6","scoping","js","advanced javascript","advanced frontend"],"slug":"understanding-temporal-dead-zone-in-javascript---rid---0KEfke08AOZuNXVxRXPW","title":"Understanding Temporal Dead Zone in JavaScript","resourceId":"0KEfke08AOZuNXVxRXPW"},{"_id":"5f1ff867cbec5f7ffc0c2fb6","content":{"difficulty":2,"domain":2,"type":1},"tags":["javascript","frontend","reactjs","profiling","performance","chrome","devtools","advanced frontend"],"slug":"profiling-react-performance-with-react-16-and-chrome-devtools---rid---Oj6vkw9ga5NIqw4266kr","title":"Profiling React performance with React 16 and Chrome Devtools.","resourceId":"Oj6vkw9ga5NIqw4266kr"},{"_id":"632544a5ff34d52bbb0796d5","content":{"difficulty":1,"domain":2,"type":2,"isInternal":false},"tags":["javascript","frontend","js","advanced javascript","localstorage","browser","web storage","cookies","session storage","ui","tutorial","youtube"],"slug":"how-to-design-and-implement-localstorage-api-or-frontend-data-structures-or-javascript-interview-questions---rid---kh86UNH6qMxBEaWBcTgr","title":"How to design and implement LocalStorage API | Frontend Data Structures | JavaScript Interview Questions","resourceId":"kh86UNH6qMxBEaWBcTgr"},{"_id":"62d3b928d3473370ac372879","content":{"difficulty":4,"domain":2,"type":2,"isInternal":false},"tags":["javascript","promises","async programming","advanced javascript","interview question","frontend","coding"],"slug":"javascript-promises-fundamentals-every-engineer-should-know-or-part-2-or-event-loop-or-microtasks---rid---oBrzK6Mt7HNAnKBT3ogC","title":"JavaScript Promises Fundamentals Every Engineer Should Know | Part 2 | Event Loop | MicroTasks","resourceId":"oBrzK6Mt7HNAnKBT3ogC"},{"_id":"627795d91195627fe9f20c27","content":{"difficulty":4,"domain":12,"type":2,"isInternal":false},"tags":["product management","video podcast","devtools tech","youtube","product manager","remote","remote job","yomesh gupta","art"],"slug":"the-art-of-product-management-or-in-conversation-with-sourabh-ahuja-or-devtools-tech---rid---Ad5B6k5Zu9Oja2NdFK3t","title":"The Art of Product Management | In Conversation with Sourabh Ahuja | Devtools Tech","resourceId":"Ad5B6k5Zu9Oja2NdFK3t"}]}}