Handling Exceptions from PIDataPipe.GetObserverEvents: Best Practices for High-Uptime Applications

Working with OSIsoft PI Data Archive's PIDataPipe and PI Update Manager can be a challenge for applications that require high reliability and near-continuous data capture. When working with the PIDataPipe.GetObserverEvents method through the AF SDK, encountering exceptions is inevitable—how you react to them can make the difference between losing critical data and maintaining a robust integration.

In this post, we'll explore the most common exceptions you may encounter using PIDataPipe, what they mean, and recommended actions to ensure your application remains as close to 100% uptime as possible.

---

Key PIDataPipe Exceptions and Their Implications

Let's examine the principal exceptions and what they signal in the context of a PI data pipe:

1. PIConnectionException

What is it? A transient error indicating a network communication issue between your client and the PI Data Archive.

Action:

• This exception does not always mean your current data pipe is dead, but it suggests heightened risk.
• Even if the PIServer object can reconnect, PI Update Manager associates sign-ups (i.e., subscriptions to data updates) with a unique connection ID. A reconnection usually results in a new connection ID, meaning previously established sign-ups may need to be recreated.
• Recommended: Always check the status of your sign-ups after recovering from a connection exception, and be prepared to re-create your data pipe subscriptions.

2. AFDataLossException: Signups Dropped / Consumer Removed

What is it? These messages occur when your registration for updates has been dropped by the PI Update Manager—commonly due to prolonged disconnects or system resource limitations.

Action:

• This is a fatal state for the data pipe: the sign-up cannot recover.
• Recommended: Dispose of the existing data pipe and create a new one. Accept that some data (events that occurred while disconnected) will not be available through the pipe.

3. AFDataLossException: DataOverflow

What is it? An overflow of the consumer queue maintained by the PI Update Manager. The queue has a limited size, and overflowing means events have been lost.

Action:

• Some tags or events are lost and irretrievable. However, you can continue to consume remaining events in the queue.
• Note that after an overflow, you might observe decreased throughput. This can be due to internal back-pressure mechanisms or resource limitations triggered by the overflow event.
• Recommended:
  • Consider recreating the data pipe if throughput is severely reduced.
  • Investigate the frequency of your calls to GetObserverEvents; polling too infrequently increases overflow risk.

---

Strategies to Minimize Data Loss and Maximize Uptime

1. Poll the Data Pipe Frequently

• Invoke GetObserverEvents as often as every 0.5 to 5 seconds. Less frequent polling increases the chance of queue overflow and lost events.

2. Tune Server-Side Queues (With Caution)

• The PI Update Manager's MaxUpdateQueue and TotalUpdateQueue tuning parameters control queue sizes per consumer and overall. If your data volumes are high and hardware permits, coordinate with your PI admin and OSIsoft Technical Support to safely increase these values.

3. Detect and Recover from Pipe Failure

• Monitor for fatal exceptions and automate the process of disposing and re-creating pipes/re-subscribing to events.

4. Understand the Limits of Persistence

• Some OSIsoft components (Alarm and Totalizer Subsystems) use persistent sign-ups that survive client interruptions. However, as of current AF SDK versions, application developers cannot create persistent data pipes in custom solutions; this feature is not exposed programmatically.

5. Monitor and Audit Sign-Ups

• Use utilities like pilistupd.exe or monitor the PI Data Archive logs to audit the status of sign-ups and track client connections.

---

Conclusion

Handling transient and fatal exceptions from PIDataPipe.GetObserverEvents wisely is crucial for high-uptime, high-integrity PI data applications. Generally, connection losses may require re-signing up for data, and overflows are best prevented by tuning queue sizes and using frequent polling. Persistent data pipes remain a feature reserved for select OSIsoft subsystems, so applications must be architected with robust error recovery and loss mitigation strategies in place.

By understanding these behaviors and proactively responding to exceptions, you can significantly reduce unplanned downtime and data loss in your PI integrations—and keep your systems drinking from the data "glass," rather than watching valuable information overflow the rim.