Apache Spark 1.1.1 Released

By CIOReview | Monday, December 15, 2014

FREMONT, CA: Apache group has announced the release of Spark 1.1.1 that has bug fixes contributed by 55 developers. Fixes are available for Spark Core, SQL, PySpark, MLlib, streaming, and GraphX.

Apache Spark which is a fast and general engine for large-scale data processing has been getting increasing adoption with the growing demand for real-time analytics solutions. Being an open source data analytics cluster computing framework, Apache Spark easily outperforms Apache Hadoop by having 10 to 100 times greater processing speed. In-memory computing capabilities of Apache Spark makes it more efficient Hadoop which implements processing that requires data to be moved in and out of disk-based storage. Another major differentiator between Hadoop and Spark is that the earlier is based on batch-processing while the latter is based on stream processing.

Influenced by the stream processing capabilities, DataStax – provider of Apache Cassandra, a distributed database technology – has adopted Apache Spark. DataStax Enterprise 4.6 (DSE 4.6), the latest database platform for IoT, web and mobile applications offers Apache Spark streaming analytics integrated with real-time transactions processing yields greater levels of personalization at global scale. DSE 4.6 enables swift development of scalable fault-tolerant streaming applications for real-time data scenarios.