LDAR Tools Inc.
In 2013, Art+Logic was hired to create a small application which would organize the routes of in-field technicians surveying every joint, valve and fixture at a massive processing location.
Later that year, LDAR asked Art+Logic to hop on a legacy project that was in need of debugging and refactoring.
Then, in 2015, LDAR contracted with Art+Logic to build out a massive compliance platform: Chateau.
Rex Moses recognized that the current solution offerings processing the data his hardware and software were collection were dated and prone to exposing clients to errors worth millions of dollars in fines or, conversely, over-monitoring. He had a vision for a feature-rich solution built upon the powerful technologies that would provide clients replicable reports, unmatched insight and control while remaining flexible and customizable to each client’s particular product’s needs.
// Goals + Objectives
However, at the time, people looking for a monitoring platform could choose, really, from only two very poor solutions. For all the importance placed on LDAR, the platforms available were written with dated technologies, failed to produce reproducable results and, in one case, was a poorly extended version of an application used, originally, to monitor fire extinguishers.
The client had a vision for a contemporary, powerful, flexible and fully featured application which would tie together all aspects of LDAR – from building out and applying new rules, to preparing government-ready reports, to tracking and approving repairs, to organizing teams to run onsite analysis and scheduling them so that a user was ensured that adequate monitoring could be completed by the staff on hand at the appropriate times.
We often hear, at Art+Logic, that an application would “revolutionize” an industry, and many times, that’s hyperbole. In this case, however, this project would actually do just that.
In 1999, the EPA estimated that noncompliance on fugitive emissions released an additional 40,000 tonnes of VOCs into the air.
The Leak Detection and Repair: A Best Practices Guide – is intended for use by regulated entities, such as petroleum refineries and chemical manufacturing facilities, as well as compliance inspectors. The guide details some of the problems identified with leak detection and repair (LDAR) programs. It focuses on Method 21 requirements, and describes the practices that can be used to increase the effectiveness of an LDAR program.
The data is supreme; everything else follows the data. Whenever data design and code design ideas are at odds, by default, data wins. Object-orientation was not maximized to its full potential, but rather was honed to facilitate data management, especially via the ORM capabilities of the Entity Framework. Object-oriented design was employed more fully in the UI.
Many activities, from data entry to scheduling inspections are affected by rules. Although some simp ler aspects of rules could be represented in the database, there are some aspects that could not or should not be evaluated in the database. This meant that the Business Engine became the centerpiece of the Chateau system.
Complex rules are evaluated in Workflow Foundation (WF) activities. Each activity processes a specific part of the rules. After the end of a flow, results are saved back to the database. These results are later used as inputs of other flows.
Given the requirement of high flexibility in the representation and configuration of rules, overly-rigid data structures would not suffice. We chose to represent the semi-structured data of rules in XML. This choice facilitated the configuration of rules in the Builder application. At the same time, we employed schema validation to ensure that rules met a minimal standard before being parsed and saved to the database.
Due to the importance of complex business rules, the Business Engine must not be bypassed; therefore, users must not access the database directly. Instead, all data access is performed via services. Similarly, third-party developers should not access the database directly. Instead, third-party developers will create service clients.
Strictly speaking, there is no need to provide libraries to third-party developers. Developers could use service discovery alone to generate proxy classes. However, as a courtesy to .NET developers, it is possible to provide a library of all the data model classes, which would save them the trouble of generating proxies.
Communication with the Business Engine is done mostly via Windows Communication Foundation (WCF) service methods. WCF supports many bindings. For improved speed and security, we chose Net.TCP by default. In some cases, namely file transfers, we use other protocols such as HTTP.
Most service methods use plain-old C# objects (POCOs) defined as data transfer objects (DTOs). In cases where variable or semi-structured data is desired (e.g. to populate a datagrid with variable columns), methods return DataSets. Most update methods return the updated object; this serves as a confirmation of the update.
Using WCF, exceptions thrown by the Business Engine are turned into faults sent to the client. This usually provides more information that other error reporting mechanisms. Exceptions are thrown whenever unexpected behavior occurs. This includes a failure to update data or a failure to find a unique item.
LTI’s clients have, in many cases, years of historical data in legacy LDAR systems. Some clients may have relatively small plants with small numbers of components; others will range into the tens of thousands. LTI would not be able to make a strong sales case for Chateau if there was no possibility of loading this legacy data into Chateau.
Loading this data, however, encountered a number of serious obstacles.
An interface was created to allow the subject matter experts at LDAR tools to analyze component type differences. This data was used to build a translation table which was integrated with the data loading process.
Candidates for duplicate data were extracted, presented to the client for approval, and then eliminated in the source queries extracting data from legacy systems.
We were able to use T-SQL’s window functions to analyze the legacy historical data in a way which isolated and tracked changes to each individual field, even though the legacy systems tracked all fields changes as a single record.
Previous applications required users to update their application by installing updates disks which was a scaling nightmare. We needed to build a Rule Building Engine which would allow users to define rules relative to their local, state and federal guidelines per their industry WHILE ensuring that the rules they created mapped to the application and created reproducible and reliable reports.
The client is an industry domain expert, having worked in the LDAR field for decades. Much of the project requirements had to be extracted, organized and transformed to project specifications from the vast knowledge base that existed in Rex Moses’s head: He knew the current workflows as well as where other creaky competitor’s applications failed or fell short to really provide today’s users with the information they needed. Although Rex and his company did document the features of the application, the many nuanced relationships of those features and the data that sat at the heart of them, needed to be teased apart one thread at a time.
For the size and ambition of the project, the Chateau team started out small with only two lead developers, designer, PM and QA. This reflected the team’s initial focus on learning the client’s vision for Chateau intimately, understanding the problem domain and building a detailed backlog and initial sequencing plan for the project.
This phase lasted through several four-week milestones and included development of prototypes and proof-of-concept code for key areas of business logic as well as fleshing out of the UI design. Though it is tempting to shortcut this phase of a project and dive into doing “real” development right away, taking the time to get a deep understanding of the project at the start saves time and cost in the long run, particularly for a more complex problem domain.
The first phase of Chateau culminated in a goal for an alpha release. This included both identifying the approximate minimum features necessary for alpha as well as a limited business environment where alpha testing would occur. The client identified schedule as the key driver for them, so Art+Logic scaled the team up heavily for the push to alpha.
It’s worth noting that this isn’t the “mythical man month” approach to making a project go faster: just add more developers. Instead the scaling up of the team was driven by Art+Logic and the client being able to create requirements for development at a more rapid pace as their mutual understanding of the project matured.
The Alpha development phase ended with a slowing of development velocity over two milestones as the parallel development efforts were completed and final integration and testing work was completed prior to the Alpha release.
Priorities changed when a regulatory change in the client’s business domain prompted a new business opportunity for them. This resulted in a change in direction and sequencing of the project for several milestones to add a new feature set to address that new opportunity.
Then it all ends with beta testing and final release.