A new press release reports, “Dremio, the Data-as-a-Service Platform company, announced today a new open source initiative for columnar in-memory analytics based on Apache Arrow. The Gandiva Initiative for Apache Arrow leverages the LLVM Project, an open source compiler, to significantly improve the speed and efficiency of performing in-memory analytics using Apache Arrow, making these improvements widely available to many languages and popular libraries, initially for C++ and Java, and eventually others including Python, Ruby, Go, Rust, and JavaScript. The Gandiva Initiative for Apache Arrow provides the following benefits to make analytical data transportable and more efficient: (1) Faster time to insight in analytics, machine learning, and data sciences. (2) Lower cost of operations on cloud infrastructure for analytics, machine learning, and data sciences.”

Jacques Nadeau, co-founder and CTO of Dremio and PMC Chair of Apache Arrow, commented, “Apache Arrow was created to provide an industry-standard, columnar, in-memory data representation… Dozens of open source and commercial technologies have since embraced Arrow as their standard for high-performance analytics. The Gandiva Initiative introduces a cross-platform data processing engine for Arrow, representing a quantum leap forward for processing data. Users will experience speed and efficiency gains of up to 100x in the coming months.”

The release adds, “LLVM is an open source project originally developed by Swift language creator Chris Lattner. LLVM’s Just-in-Time compilation capabilities incorporate runtime information to produce highly optimized assembly code for the fastest possible evaluation. By combining LLVM with Apache Arrow libraries, low-level operations on Apache Arrow in-memory buffers such as sorts, filters, and projections can be highly optimized for specific runtime environments, improving resource utilization and providing faster, lower-cost operations of analytical workloads.”

Read more at Business Wire.

Photo credit: Dremio