DISTRIBUTED PROCESSING

Geting the most out of akka clusters

SURAJ ATREYA
Aug 14, 2015

Anyone serious about distributed systems or building one, commonly encounters issues such as replication, consistency, availability and partition tolerance (CAP) [1]. In a real life scenario, partition tolerance is inevitable. So the system must be able to handle partition tolerance when there are network outages. Therefore, ‘P’ in the CAP is a must for any distributed system. This has been backed by Peter Deutsch in his (EIGHT FALLACIES OF DISTRIBUTED COMPUTING).

Elastically adding and removing nodes using akka cluster

SURAJ ATREYA
Aug 07, 2015

This post explores a pull-based master/worker architecture – one that is suitable for anyone who is looking to elastically provision nodes when the load is higher than normal and under-provision when the load is below normal. In this post, the master accepts RSS links from frontend which can be a user submitting links and worker accept one RSS link and extract information such as article’s published date, title and a brief description. All this information is indexed into ELASTICSEARCH.

Refining the art of query performance

ASHOK AGARWAL
Oct 26, 2014

Ever wonder how we power those “which controller went down today” queries that sprawl 1000s of databases, amounting to 100s of terabytes of log data every day? How do we deal with terabytes of data in a robust and efficient manner? We call it harmonic in memory query management.

We’ve been working with a distributed Cassandra cluster for almost a year. During that time, we have learned a bit about achieving scalability, and along the way we have collected some insight on achieving optimal query performance.

Home Section banner