Apache Spark is growing, but shortage of Spark skills is a significant adoption barrier
Apache Spark is one of the most popular big data frameworks today. It has moved beyond the early-adopter phase and is now mainstream in large data-driven enterprises. Our customers, primarily Fortune 500 companies, are looking at Spark for all data processing tasks. These range from ingest through ETL and data quality processing to advanced analytical tasks and machine learning jobs.
Even though Spark’s popularity has grown significantly, unavailability of Spark talent is impacting a wider and deeper adoption of Spark.
Why Apache Spark skills are not growing at the same pace
With the rapidly changing big data technology landscape, Spark itself is evolving, and developers and enterprise IT teams can find it challenging to keep up with the pace.
Moreover, though Spark’s open source availability provides an easily accessible platform for experimentation, it demands a steep learning curve. It also requires a lot of development, integration and testing time to write code and solve the complexities of building Spark-based production-ready applications.
Simplifying Apache Spark is the answer
StreamAnalytix Lite provides a solution to the complexities involved in building enterprise-grade applications on Spark for both batch and streaming mode. It is a free, quick-start lightweight product, which anybody can download and use to accelerate their Spark learning and usage.
A Spark only version of StreamAnalytix (an open-source enabled, enterprise-grade, stream processing and machine learning platform), StreamAnalytix Lite offers the same powerful visual interface that dramatically increases developer productivity by providing ready-to-use operators to select, drag-and-drop, connect, and configure to realize a fully functional Spark pipeline.
It can receive data from a wide array of local data sources and data targets and offers all the advanced analytics and machine learning capabilities of StreamAnalytix on a single instance.
StreamAnalytix Lite offers the following key features:
|A GUI-based development and operations tool:|
Use it to learn, experiment, develop, and put Spark applications into production
|A full portfolio of debugging, administration, and monitoring functions:|
Build, test, run, and manage Spark applications end-to-end
|Extremely easy to work:|
Lightweight with 2GB on disk, can be downloaded onto your Windows, Mac, or a Linux desktop or a server node
|No need for coding, yet enables custom logic:|
Comprehensive set of pre-built big data tools and drag and drop operators
|A web-based tool with powerful multi-tenancy features:|
Allows multiple users to connect to a single node
|StreamAnalytix Lite Interface|
It offers a powerful visual interface with a pipeline designer for rapid application development and built-in dashboards for real-time data visualization.
Detailed list of in-built Apache Spark Operators in StreamAnalytix Lite
Operators include an array of data sources, processors, analytical operators, and emitters
Recommended usage of StreamAnalytix Lite
Though StreamAnalytix Lite makes Apache Spark development easy, it is not recommended as an execution platform for production applications. For this, pipelines built on StreamAnalytix Lite can be seamlessly exported to the production-grade (eEnterprise) edition of the StreamAnalytix platform to run at full enterprise scale in production on multi-node Spark clusters.
Support for StreamAnalytix Lite
Developers who may need some help or support can also consult with StreamAnalytix experts and a forum of peer-developers on the web-based community portal for this tool. Similarly, experienced developers can also contribute their knowledge or sample pipelines. Click here to reach Support Forum.
Meetup – Stream Processing with Apache Kafka, Flink and Spark
November 2, 2019 - November 2, 2019 Hotstar office (Bangalore)
Anomaly Detection with Machine Learning at Scale (India)
Sep 27, 2019 | 11:30 am IST