Pipelining is the technique used by HDFS to minimize inter-node network traffic. Whenever the first block replica is written by a to a node, then it’s the responsibility of that node to write the second replica to a random off-rack…
Read More HDFS Pipelining to minimize inter-node network traffic
Apache Hadoop was initially developed by “Doug Cutting” in 2005 because he needed a faster data processing framework for the web crawler project called Nutch. Based on the MapReduce paper which was published by Google in 2004, he replaced the…
The DocumentDB Data Migration Tool is an open source tool for importing data to DocumentDB from a variety of sources such as JSON files, CSV files, MongoDB, SQL Server, Azure Table storage, Amazon DynamoDB, HBase etc The import tool can…
Read More Migrating data from different sources to DocumentDB
Amazon Kinesis Firehose is a managed service to load real-time streaming data to Amazon Simple Storage Service (S3), Redshift or Elastic search Service (ES). Firehose is part of the Amazon Kinesis streaming data platform, along with Amazon Kinesis Streams and Amazon Kinesis Analytics. There…
Read More Amazon Kinesis – Managed service for real-time data processing
Download Presentation There are many different queuing systems available like Kafka, MSMQ, RabbitMQ, Amazon SQS etc. Azure Service Bus Brokered Messaging is a queuing system that is a scalable and feature-rich messaging service hosted in Microsoft Azure, or available as…
Near real time Big Data from social network sites like Facebook and Twitter has been an interesting source for analytics by researchers in recent years due to various factors including its availability and popularity, though there might be a…
Read More USING SPARK FOR STREAMING TWITTER DATA FOR EFFECTIVE JOB SEARCH
The big data can play a vital role in medicine building better predictive models around individual patients resulting into better diagnosis and treatment of disease. The last decade has seen huge advances in the amount of data we routinely generate…
The interquartile range of an observation variable is the difference of its upper and lower quartiles. It is a measure of how far apart the middle portion of data spreads in value. When a data set has outliers or extreme…
The lack of communication between statisticians and the managers is the major roadblock for using statistics. Below are some statistical terminology to ease a precise communication Population: A population is any entire collection of people, animals, plants or things on which…
We can break up any statistical problem into three steps: Data collection and Sampling. Data analysis. Decision making. It is well understood that step 1 typically requires some thought of steps 2 and 3. It is only when you have…
Read More Relevance of decision-based thinking in statistical analysis
