× {{alert.msg}} Never ask again
Get notified about new tutorials RECEIVE NEW TUTORIALS

Optimizing the design of a backend search engine

Mar 11, 2015
<p>Summary from comments:</p> <ol> <li><p>You don't need Redis for the caching as EL takes care of all those and it's scalable the same way. <a href="http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html" rel="nofollow">http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html</a></p></li> <li><p>You should not store the entire HTML, you don't need that phase, as EL responds in under 50ms with your results.</p></li> <li><p>For hot/trending static pages, you could have easily a Varnish before your chosen web server, and that could carry 10k ops/sec</p></li> <li><p>Use CDN, Only if CDN costs are less than paying for a instance which can generate it on the fly. </p></li> <li><p>Don't put in the same bucket time of templating and time of accessing from storage. While the first one I think will be under 1ms, the later - on a good cluster could be 50ms. Maybe you will have another 50ms from all the other routes (DNS, load-balancer, logging), so I think that this way you can serve under 110ms the whole static thing.</p> <ul> <li>if you will go with Elasticsearch, you don't need MongoDB at all, as EL is also a document storage.</li> </ul></li> </ol> <p>This tip was originally posted on <a href="http://stackoverflow.com/questions/28705251/Optimizing%20the%20design%20of%20a%20backend%20search%20engine/28805280">Stack Overflow</a>.</p>
comments powered by Disqus