Peak ODS Adapter for Apache Spark: Scalable data access based on Apache Spark
To gain more insights from your existing measurement data stored in your ASAM ODS data management system, Peak Solution has developed the Peak ODS Adapter for Apache Spark. The Peak ODS Adapter for Apache Spark allows you to access instance and bulk data with Apache Spark SQL and DataFrames.
Using Jupyter Notebooks
Apache Spark provides several high-level APIs such as Python and others that can be perfectly used in Apache Zeppelin or Jupyter Notebooks for interactive analysis of measurement data. The returned DataFrames can easily be used for further analysis and statistics of the data.
Scalability and big data support
An important advantage of the Peak ODS Adapter for Apache Spark is its scalability. The implementation of the Peak ODS Adapter for Apache Spark enables complex data queries and data analyses to be performed on large amounts of data at high speed. Depending on the number of licensed executors and the performance of your Apache Spark cluster, you can scale from local (standalone) to high-performance clusters. In addition, the Peak ODS Adapter for Apache Spark implements the ASAM ODS Big Data Connector - a standardized way to export content from your Peak ODS server in the form of AVRO or Parquet files. These exports can be integrated into existing big data ecosystems such as Apache Hadoop. This means that companies can use the Peak ODS Adapter for Apache Spark within their existing ODS infrastructure without extensive IT investments and gradually expand it into a powerful big data cluster for managing and analyzing large volumes of test data.
Contact us if you would like to learn more about how Peak ODS Adapter for Apache Spark can support your test data management solution.
Connected solutions
You can click on the links to get more information about the individual components
Peak Test Data Manager
Peak Test Data Manager is a future-oriented test data management system and uses Peak ODS Adapter for Apache Spark for scalable data analysis.
Peak ODS Server
Implements the ASAM ODS data standard for long-term data storage including APIs for standardized and secure data access.
Related topics
What is ASAM ODS?
The ASAM ODS standard defines APIs and formats for storing and retrieving test and measurement dataInformed decision making with data analysis
Overview of data analysis options and the use of available analysis tools.
Python ODS utilities
Open source libraries and examples for using ASAM ODS data in Python.
What are Jupyter Notebooks?
