Deep Learning Based Automated Right-of-Way Surveillance

Introduction
Rapid advances made in the area of camera and sensor technology has enabled the use of video acquisition systems to monitor the right-of-ways (ROW) of oil and gas pipelines. Huge amount of data is thus made available for analysis. However, it would be very expensive to employ analysts to scan through the data and identify threats to the right-of-ways in the vast amount of wide area imagery. This warranted the deployment of an automated mechanism that is able to detect threats to the right-of-way and send out warnings in the event of detection of a threat. The current research in the area of neural network based deep learning methodologies for object detection, recognition, identification, and tracking has shown significant progress in the performance of computer vision applications. Deep learning is a machine learning method based on learning data characteristics and representations. Deep learning models are inspired by the structural and functional properties of the biological nervous systems. The deep learning based computer vision system adopts the visual perception behavior of the human visual system. We propose to develop a software package for automated threat detection on pipeline right-of-ways by employing the advanced deep learning based artificial intelligence technologies. The main objective of the proposed pipeline right-of-way surveillance system and threat recognition software is to identify, localize, and track potential threats to the pipeline right of ways. This will be achieved via deep learning based artificial intelligence algorithms, which will analyze high-resolution imagery captured by cameras located on small aircrafts flying over the pipeline right-of-ways.

Goals/Objectives
Developing a real-time threat detection solution that will identify and display potential threats to the operator along the right-of-way during the survey operation

Methodology
The proposed deep learning based artificial intelligence methodology for machinery threat detection on pipeline right-of-ways include several computational units viz. key frame selection, background region elimination, object feature extraction, object detection, object localization, and object tracking. The overall deep learning architecture for the proposed threat object detection system is shown in Figure 1.

 
Figure 1. Deep learning approach for object detection and classification.

We propose a neural network architecture named as the deep perceptual neural network (DPNN), which is a convolutional neural network optimized for detection and classification. The network is comprised of four different parts: backbone, feature fusion, classification, and regression. These parts of the DPNN work together for extracting relevant features and determining the bounding box and class of the detection.

Results/Evaluations

We have used the Aberdeen dataset collected by Pipeline Research Council International (PRCI) as an initial dataset for testing and evaluation. The dataset has 4380 images with a ground sample distance of ~3 inches. Figure 2 shows some images of the detections made by the DPNN algorithm. It is found that the DPNN works very well in detection and classification of the machinery and vehicles in the dataset and will generalize fairly well in different environments.
 
Figure 2. Initial detection results of machinery and vehicles on the PRCI Aberdeen dataset

We have also conducted initial evaluations of the DPNN architecture using the vehicle detection in aerial imagery (VEDAI) dataset, which has several vehicle classes and multiple complex scenarios. This dataset is lower resolution than the given PRCI data. Figure 3 shows various detection on the VEDAI dataset using the proposed DPNN architecture. It is found that the DPNN architecture provides accurate detection and classification of vehicles in both simple and complex backgrounds.

Figure 3. Initial detection results of different classes of vehicles on the VEDAI dataset