Automation applied to technical drawings in industry: Interview with Vasil Shteriyanov

In their paper Automating the Expansion of Instrument Typicals in Piping and Instrumentation Diagrams (P&IDs), presented at The Thirty-Seventh Annual Conference on Innovative Applications of Artificial Intelligence (IAAI 2025), Vasil Shteriyanov, Rimma Dzhusupova, Jan Bosch and Helena Holmström Olsson focus on automation of technical drawings in industry. In this interview, Vasil tells us more about their work.

What is the topic of the research in your paper?

Our paper focuses on automating the Instrument Typical Expansion process in Piping and Instrumentation Diagrams (P&IDs), which are vital technical drawings used in the engineering, procurement, and construction (EPC) industry. P&IDs are used to represent the layout of piping systems, instruments, and other equipment in large-scale infrastructure projects.

A key challenge with P&IDs is that they often include Instrument Typicals – simplified representations of standard instrument assemblies, rather than visualizing each individual instrument. These actual assemblies are found in legend sheets visualizing all instruments (Figure 1).

Figure 1: A legend sheet excerpt depicting the simplified representation of instrument assemblies shown in the P&IDs, and the detailed representation of all utilized instruments.

Instrument Typical Expansion refers to the process of taking these simplified representations in the P&ID and expanding them by referencing the legend sheet to fully enumerate the instruments contained in each typical. This process is traditionally manual, requiring engineers to look up the corresponding legend information and populate the Instrument Index – a comprehensive document that lists all the instruments used in a project, along with their details. This manual method is particularly time-consuming during the tendering phase of a project, when engineers need to analyze multiple P&IDs to generate accurate cost estimates and project deliverables.

In our research, we propose a method that automates this process using computer vision models and domain-specific rules. Our approach extracts both the text from the P&IDs and the structured information from the legend sheets, then uses this data to automatically expand the Instrument Typicals. The result is an automatically generated Instrument Index, which accurately lists all the instruments in the project without the need for manual input. This method not only streamlines the Instrument Typical Expansion but also reduces errors and saves significant time in generating critical project documents, ultimately enhancing the efficiency of project planning and execution.

Could you tell us about the implications of your research and why it is an interesting area for study?

The implications of this research are significant for the EPC industry, particularly in automating traditionally manual processes that are both labour-intensive and prone to error. During the tendering phase, engineers must analyze P&IDs to generate cost estimates. This process involves identifying Instrument Typicals in the P&IDs and manually referencing the legend sheets to determine the full set of instruments that make up each typical. The manual nature of this task leads to inefficiencies and potential errors that can have serious consequences later in the project.

Our proposed method automates the Instrument Typical Expansion process by combining advanced AI models for text detection with domain-specific engineering knowledge. This could significantly reduce the time spent on these tasks, improve the accuracy of project documents, and ultimately reduce costs for EPC companies. Furthermore, by streamlining this process, engineers can focus on higher-level tasks, improving the overall efficiency of project execution. The research also opens the door to further automation possibilities in engineering design, especially as companies increasingly move towards digital tools and AI-driven workflows.

Could you explain your methodology?

The methodology for automating Instrument Typical Expansion (Figure 2) involves three key stages that integrate AI-based text recognition and engineering-specific rules:

• Text Detection and Recognition: The first step involves detecting and recognizing all text within the P&IDs and legend sheets. We use a trained text detection model tailored to internal P&ID data, capable of handling the dense and complex layouts typical of engineering drawings. To improve detection of small and overlapping text elements, we apply a tiling strategy that divides each P&ID into overlapping regions. For text recognition, we use a pretrained text recognition model, enabling accurate transcription of text elements across varied drawing formats.
• Legend Information Extraction: Instrument Typicals are fully defined in legend sheets, where each typical is represented with both a simplified schematic and a list of detailed instruments. To extract this information, we first detect the boundaries of each typical using line detection methods. These boundaries help us group text data into distinct sets corresponding to each typical. We then apply domain-specific rules to group the extracted instrument information for each typical.
• Generation of Utilized Instruments: After identifying the Instrument Typicals in the P&IDs and extracting their definitions from the legend sheets, the system automatically expands each typical into its full list of instruments. It uses spatial and textual cues to ensure the correct mapping between the P&ID and legend data. Optional instruments are included only if present in the P&ID. The resulting expanded instruments are compiled into a complete and accurate Instrument Index document.

Figure 2: A visualization of the proposed method for automatic Instrument Typical Expansion in P&IDs.

What were your main results and how did you evaluate the method?

We tested our method on 55 P&IDs and 4 legend sheets from three real-world EPC projects property of McDermott. These images were not used in training the model to ensure unbiased results. Our evaluation yielded impressive results:

• 100% accuracy in extracting Instrument Typical groups from the legend sheets.
• Over 98% recall and 99% precision in generating the final list of instruments from the P&IDs and legend sheets across all projects.

These results demonstrate the effectiveness and reliability of the method in expanding Instrument Typicals, with errors primarily occurring due to missed text detections or incorrect instrument associations due to text proximity. However, these errors were rare and did not significantly affect the overall accuracy, confirming the method's applicability for industrial deployment, particularly in large-scale projects where precision is essential.

Could you talk about your path to deployment for this method?

Following positive feedback from a limited group of instrumentation engineers, we are working on integrating our method into an internal application used to generate Instrument Index documents.

Our method requires GPU support due to its reliance on deep learning models for text detection and recognition. Preliminary testing has demonstrated that the method can process up to 100 P&IDs in under an hour, drastically reducing the time spent on manual expansion. This efficiency, especially during the tendering phase, can lead to significant cost savings.

What further work are you planning in this area?

We are planning several directions for future research and development. One key area exploring the possibility of making the method more adaptable to different formats, particularly since the current approach is customized for McDermott’s specific legend sheet format. A format-agnostic approach would allow the system to handle different formats used by various EPC companies.

About Vasil

Vasil Shteriyanov is an industrial PhD student at Eindhoven University of Technology, focusing on automating manual engineering practices using AI. He holds a Master's degree in Data Science and Artificial Intelligence and a Bachelor's degree in Computer Science from Eindhoven University of Technology. His PhD research is closely tied to his role at McDermott, a global Engineering, Procurement, and Construction (EPC) company, where he works as an AI Researcher and developer. His research aims to develop innovative solutions that streamline and enhance traditional engineering workflows through advanced artificial intelligence techniques. At McDermott, Vasil is involved in a variety of AI projects that leverage deep learning for computer vision and information extraction from engineering drawings. His work involves close collaboration with cross-functional teams to ensure that their AI solutions effectively meet the specific needs of different engineering departments

Read the work in full

Automating the Expansion of Instrument Typicals in Piping and Instrumentation Diagrams (P&IDs), Vasil Shteriyanov, Rimma Dzhusupova, Jan Bosch, Helena Holmström Olsson.

Lucy Smith is Senior Managing Editor for AIhub.

Lucy Smith is Senior Managing Editor for AIhub.