Integration of Apache Hadoop
Possible use of Apache Hadoop for long-term storage of iba data
Industrial production generates a large amount of different technical operating data. This group also includes process data, measurement data and metadata. The process data of a plant are usually acquired cyclically, while measurement data can belong to a process as well as to a product or batch. Metadata, such as product numbers, customer numbers, or other additional information will also be added. Technical operating data can be recorded, stored, analyzed and visualized with the iba system. In general, detailed data recording is necessary for quality documentation. It is also useful to identify potential in process and quality improvement.
The amount of data generated in a company increases rapidly with its size and level of digitalization. The effort and complexity of data management also increase. Therefore, this white paper explores the question of how users of iba software can handle such requirements and what solutions are available to manage very large amounts of data. For this purpose, investigations were carried out at iba as part of the NewTech4Steel research project. This white paper includes basic information about Apache Hadoop, the results of the project, and recommendations for iba users who have an enormous amount of data or plan to use Apache Hadoop to store and analyze measurement data. It also follows on from the previous white paper regarding the ibaHD-Server benchmark. The results of the ibaHD-Server benchmark have shown that ibaHD-Server is very well suited for high-frequency data recording due to its efficient storage algorithm. The focus in this document is now on scalability and solutions for big data.
Download:
You can download the white paper "Integration of Apache Hadoop" here.
|Miscellaneous Retour