Session: 06-09 Advanced Simulation & Testing

Paper Number: 129443

129443 - A Large Language Model Interface for Cycle Modeling 

The Numerical Propulsion System Simulation (NPSS) is a modular and extensible thermodynamic modeling environment used by the aerospace industry for modeling turbomachinery, air-breathing propulsion systems, liquid rocket engines, engine control systems, and system model integration. It can also be used for modeling refrigeration cycles, multi-phase heat transfer systems, vehicle emission analyses, or supercritical carbon dioxide (sCO₂) power cycles. Historically, the NPSS interface and usability, while powerful, have been predominantly code-centric, demanding users to possess not only domain-specific knowledge but also proficiency in the technical intricacies of NPSS itself. This dual requirement can sometimes act as a barrier for new users, and the complexity can also make it difficult to understand models developed by other engineers.

The rapid evolution of artificial intelligence (AI) and machine learning technologies touches many fields, including aerospace engineering. One of the most significant advancements in this domain is the emergence of large language models (LLMs) that can comprehend, generate, and interact with human language in an unprecedented manner. Products like OpenAI's chatGPT and Meta's Llama 2 exhibit human-level performance on a wide range of tasks.

This study aims to develop and evaluate an innovative LLM interface specifically designed for the Numerical Propulsion System Simulation (NPSS), aiming to harness the capabilities of an LLM to offer users a more intuitive and interactive experience when working with NPSS.

The envisioned system would, at minimum, allow users to converse with the LLM in natural language. The LLM would be equipped to intelligently discuss NPSS in general, as well as any specific model in development. The LLM implemented for demonstration will likely be a version of OpenAI’s GPT-4, but the actual implementation will allow for drop-in of local LLMs in order to protect sensitive data. Instead of tediously searching through documentation or seeking expert advice, users will be able to simply pose questions to the LLM, which, in turn, would provide insightful responses based on its extensive knowledge of NPSS. To provide this information, user manuals and other documentation will be provided to the LLM to ensure grounded responses with minimal "hallucinations" or falsehoods.

Preliminary results have demonstrated that an LLM can be conversant in the details of NPSS. Once this interface is completed, the project will attempt to extend to LLM's capabilities to actually editing code, running NPSS, and postprocessing data. To ensure safe operation, all LLM actions will be manually approved by the user, decreasing the likelihood of harmful or inaccurate operation.

To evaluate the developed system, a variety of qualitative and quantitative metrics will be employed. Users will be surveyed on the overall usability and efficacy of the tool. In addition, the tool will be evaluated on a battery of tests on NPSS theory and specific cycle modeling problems. It is anticipated that, due to the stochastic nature of LLMs, hallucinations or falsehoods will be encountered. These will be examined in order to develop deployment tactics to minimize their occurrence. Due to the safety concerns with applying a stochastic artificial intelligence tool to thermodynamic cycle modeling, outputs from this system will not be used in real-world applications at this time.

Presenting Author: Brian Connolly Southwest Research Institute

Presenting Author Biography: Dr. Brian Connolly is a Research Engineer at Southwest Research Institute. His research areas include exotic combustion modeling, multiphase flows, and large language models for engineering applications.

Authors:

Reese Roddy Southwest Research Institute
Cole Replogle Southwest Research Institute
Brian Connolly Southwest Research Institute