Automated Real-Time Process Data Analysis Using Online Wavelet Transforms : Fault Detection, Data Cleansing and Compression


Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Referred Conference Paperpeer-review

1 Citation (Scopus)


Improvements in sampling methods and sensor technology have resulted in the creation of vast amounts of process data that can be analyzed to improve the operation of chemical facilities. Automated real-Time data analysis methods sort through incoming data immediately, aiding in real-Time decision making and preparing the data for further analysis. In this study, the application of online wavelet transforms (OWT) to several aspects of real-Time data analysis was considered, namely, fault detection, data cleansing and data compression. In the first application, the OWT was used to build a model and detect faults in an uncorrelated tag (variable) which cannot use traditional data driven modelling that relies on underlying relationships between variables. The OWT was successfully able to detect and identify various common sensor faults including missing and frozen values, linear drift and spikes. Recursive least squares (RLS) was then used to predict replacement values, and was a significant improvement on current standards, particularly with highly variable data. The new algorithm (OWT and RLS together) also updates statistics with each new value, so decision making is accurate but does not require repeated calculations on the whole data set. This application directly affects real-Time decision making as detected faults can be addressed immediately, and any predictive models built from the data will be representative of real operation. The OWT is also used to create a novel real-Time data compression algorithm to economically store large quantities of data. The industry standard swinging door algorithm requires extensive manual input, in addition to other drawbacks. The new method is an online filter that is completely automated, with calculations unique to each tag. This new algorithm showed much higher compression ratios with less loss of information, and was clearly superior to the old method.
Original languageEnglish
Title of host publicationSociety of Petroleum Engineers - SPE Intelligent Energy International Conference and Exhibition
PublisherSociety of Petroleum Engineers (SPE)
Number of pages13
ISBN (Print)9781613994597
Publication statusPublished - Sept 2016
Externally publishedYes
EventSPE Intelligent Energy International Conference and Exhibition 2016 - Aberdeen, Scotland, United Kingdom
Duration: 6 Sept 20168 Sept 2016


ConferenceSPE Intelligent Energy International Conference and Exhibition 2016
Country/TerritoryUnited Kingdom
CityAberdeen, Scotland


Dive into the research topics of 'Automated Real-Time Process Data Analysis Using Online Wavelet Transforms : Fault Detection, Data Cleansing and Compression'. Together they form a unique fingerprint.

Cite this