Utility companies are under pressure to provide reliable systems, improve safety, reduce costs, and increase overall efficiencies. To reduce the amount of time and resources needed to achieve this, a machine learning based asset and infrastructure analytics platform is to be developed. This platform must be capable of leveraging remote sensing data in a variety of forms, be organized in workflows to support key utility business operations, and improve the frequency of assessment while enhancing safety.
At the heart of this platform is an analytics engine that supports a range of location detection, object identification, anomaly detection, and temporal modeling analytics. In addition, a user review workflow will be made available to assess analytics results and provide feedback for existing deep learning models.
My responsibilities were to oversee and execute the UX and product design for the platform beta as well as develop concepts for future functionality.
Role
UX and Product Design Lead
Deliverables
Flow Diagrams and Wireframes
Client
Harris Geospatial
Thumbnail Concepts
Sketching allowed for exploring many ideas and concepts very quickly. It also allowed for the elimination of bad ideas and concepts just as quickly. Thumbnails were used in all aspects of the project and throughout the design process, while mainly being used for rough wireframes and basic interaction design.
Workflows for Asset Selection
The selection of assets (i.e. utility poles), as well as the display of asset alerts, warnings, and layers are tied to user selectable workflows. These workflows support specific utility business operations:
- Inspection
- Vegetation
- Site Assessment
- Storm Assessment (Pre and Post)
Heat Maps
Utility companies manage thousands of assets in their infrastructure that are spread over a large geographic service area. Identifying problem assets quickly is vital for timely service and repair. To increase the speed of identification, the user is first shown a heat map of the utility company’s entire service area. The heat map uses a clustering methodology to allow analysts and technicians to easily see the locations and concentration of assets in their service area with the most critical issues. The heat map disappears once the map zoom level reaches a sufficient level to display asset markers and their associated details.
Information, Alerts, Warnings, Filters, and Layers
The display of infrastructure information and the totals for alerts and warnings are relative to the map zoom level and viewport. Thus the user is able to focus on the issues of a specific area while staying within the context of the total number of issues for the entire service area. The user is able to further refine what is being displayed by activating filters and layers. Information layers such as vegetation encroachment, outages, weather conditions, road access, floodplains, and more are made available to specific workflows.
Source Data and Deep Learning
Deep learning analytics are run on a convolutional neural network, commonly used for analyzing visual imagery. This imagery can be sourced from drones, satellites, digital cameras, and even mobile phones. Along with these optical sources, other remote sensing options such as LiDAR and Synthetic-Aperture Radar (SAR) can be employed. These data sources and the analytics engine are key to the overall gains in the frequency of assessment, improved reliability, reduced costs, and increased safety.
Drone Image
Satellite Image
LiDAR Scan
Reviewing and Updating the Deep Learning Models
Deep learning models are built by using examples and must be “trained.” Even though a given model may have a high percentage of accuracy, there still may be errors or opportunities to improve; this is when a workflow for assessing the analytics results occurs. A user then has the ability to remove an incorrect detection and provide feedback to the model by manually indicating what a correct detection looks like in the image through the user interface.
Two Paths for Search
A detailed image search function provides an additional way for users to locate assets with particular criteria. Two concepts were designed, each using a different methodology for conducting a search. The first concept uses a tag method where the user would select from a list of pre-existing tags; the selected tags would then be used as the parameters for the search. The second concept uses a binary method where the user would create filters using a binary statement (i.e. is, is not). These filters would then be applied to the search.
Tag Concept A
Tag Concept B
Binary Concept A
Binary Concept B
The Final Steps
Once an issue has been detected and verified, a work order can be placed in the utility company’s work order management system. A crew is then sent to the affected asset, or assets, for service or repairs. Any subsequent data or reports can then be uploaded into the system to create an asset history.
