Backfilling Historical Data from OPC HDA to OPC DA in PI System

Backfilling Historical Data from OPC HDA to OPC DA in PI System

In the realm of industrial data systems, ensuring high availability and accurate data recovery is crucial to maintain operational integrity. A common challenge that system engineers face is merging or backfilling historical data from OPC HDA (Historical Data Access) sources into DA (Data Access) PI Points within the PI System. This task is especially relevant when handling extensive data sets, often involving thousands of data point pairs.

Understanding the Scenario

In situations where data is collected in real-time from an OPC DA server, a corresponding PI Point is configured to store this continuous stream. However, there are scenarios, particularly during complete network outages, where only historical data from SCADA systems - which don’t immediately align with the real-time data - is available through OPC HDA. This necessitates having two separate PI Points initially, one for real-time data (OPC DA) and another for historical data (OPC HDA), due to differing ItemIDs.

The Key Challenge

The primary challenge faced in such scenarios is efficiently merging or backfilling the historical data collected in the OPC HDA PI Point into the existing DA PI Point. This becomes cumbersome especially when dealing with a large scale, involving potentially 2000-3000 PI Point pairs.

Solutions for Backfilling Data

1. Using AF SDK: For those with programming skills, AF SDK (Application Framework Software Development Kit) offers a robust solution to automate data transfer. This involves writing scripts or applications that can iterate over data point pairs and facilitate the transfer of historical data into the corresponding DA PI Point.

2. Employing AF Analytics: AF Analytics provides a more visual approach, suitable for those who prefer minimal coding. By configuring recalculations within AF Analytics, users can efficiently map and transfer data from OPC HDA to OPC DA PI Points. While this may require some setup, it avoids coding altogether.

3. Using PI-to-PI Interface: This approach is beneficial for automating data transfer across PI servers. Although primarily used for transferring data across different PI Data Archives, it can be configured to backfill historical data internally between PI Points under certain conditions.

Considerations

• System Redundancy: It is vital to configure system redundancy and failover mechanisms at various levels, from OPC servers to PI Interfaces and PI Servers, ensuring uninterrupted data recording and availability.

• Scalability and Maintenance: Any chosen solution should be scalable to handle large numbers of data point pairs and maintainability should be considered to make future updates and checks systematic.

Conclusion

While merging historical data from OPC HDA into OPC DA points initially appears complex, several strategies are available to simplify the task. Leveraging any of these methods can significantly enhance data continuity and ensure a comprehensive data set for analysis and reporting. For extensive implementations, understanding and utilizing tools like AF SDK or PI-to-PI Interface can save time and increase system reliability.

For further guidance, exploring OSIsoft’s extensive resources or connecting with experienced PI System professionals can provide invaluable insights.