The Future of engineering goes from numerical simulation to neural networks

Investment in different types of AI across all parts of the engineering industry is booming. Ever since ChatGPT 3.5 captured universal attention in 2022, there has been much discussion about the likelihood of AI having a transformational effect across most industries.

The advanced engineering industries are no exception, with ‘geometric deep learning’ having potential for the biggest impact.

The complexity and interconnectedness of all types of engineering systems have exploded in the last 20 years. This is exemplified by increasingly integrated systems like electric vehicles and autonomous aircraft. However, there has not been a corresponding revolution in the tools used to design and simulate these systems, until now.

Some companies are already ahead and are benefiting from embedding AI into their development processes. But often when we speak to leaders from industry they are still uncertain about how to get started, and whether they have the right foundations in place. The best way to address this is to contrast a traditional development approach with some of the new AI-driven options available that can turbocharge engineering processes.

Different flavours of AI for engineering

There are two approaches to AI that are specifically relevant to product development and the types of challenges involved in working with engineering geometry. The easiest one to understand is often called a Gaussian Process (GP).

Let's take the example of a wing. If we were to draw it in CAD we would use a handful of parameters to describe the geometry such as thickness-to-chord ratio, angle of attack and length. We could then use some simulation software to find the corresponding lifting force. If we had a handful of these data points we could train a GP (or some other AI model) to predict the mapping between input parameters and the output performance metric “lift”.

GPs can be incredibly useful and are relatively simple to train and run, but they have several limitations. We would argue the most important limitation is that for geometry problems they are relatively simplistic because they may be overlooking significant factors and details that are critical to performance but are not captured in the parameterisation.

The second approach is using Neural Networks to learn based on an entire mesh of points that represent a geometry such as a wing. As with the GP, some simulation software could be used to get a ground truth estimate of the lifting force generated by the wing. The combination of the mesh data and resulting lifting force calculated using traditional numerical simulation forms the training data for a variety of architectures of Neural Network and allows us to predict a much more generalised mapping between ‘wing shape’ and ‘performance’.

This second approach efficiently uses training data by enabling the model to learn the underlying physical effects from the entire mesh's data. It also represents a radically new way to tackle some of today’s hardest engineering problems.

From days to seconds

Let's compare this to a traditional engineering workflow using numerical simulation. Each run of CFD or FEA could take hours or days to run which would limit the number of iterations that are possible to assess. As a result, designs often fall short of their full potential because the global optimum is rarely achieved within the time available in the project.

Using geometric deep learning, each new result would arrive in less than a second (assuming the model had already been trained). This speed lends itself to automated optimisation workflows where many geometries are assessed in a short period of time using the sub-second prediction time of the AI model. As a result, we achieve better designs and time-savings that can be banked in order to bring the product to market faster.

For some categories of problem, the interactions between multiple physical phenomena are critical. A good example might be the design of a hypersonic aircraft where aerodynamics, structures, and thermal dynamics all interact. Traditional numerical simulation is usually limited to modelling one type of physics at a time, which would be time consuming and is often prone to overlooking important interactions.

However, by using geometric deep learning it is possible to train a model that has learnt the interactions between all three types of physics and can directly give accurate predictions across all of them.

Making existing data do the work

Surprisingly little is required to get started. Because geometric deep learning methods are trained across an entire mesh of data for each training geometry they are typically considered extremely data efficient (especially compared to GPs). A good model can be trained with as few as 60 matched pairs of input mesh and simulation results. The simulation data is nothing fancy. It can be created using whichever numerical simulation tools (such as CFD and FEA) an engineer is most familiar with.

Many companies already have suitable data stored away from previous iterations of their regular design work. These methods are great for putting that dormant data to good use. If you have not been storing results from previous simulation runs, there are other options which can reduce the need for artificially generating training data even further. A lot of investment is currently going into building pre-trained foundation models for engineering — which perform a similar task to Large Language Models for other applications of AI. They can act as a “warm start”, and can then be fine tuned using a small quantity of additional simulation data.

Alternatively, models can be initiated with smaller quantities of data at the expense of accuracy. However the most sophisticated models offer the user feedback by way of “uncertainty quantification” which are essentially the error bars on any prediction. If a model detects that the uncertainty is above a couple of percent, they can be set up to run what is called an Active Learning Loop, where additional training data is generated but only specifically in the areas that are most interesting and have the highest performance.

The future of engineering lies in the successful integration of geometric deep learning into design and simulation workflows. This shift offers the potential for unprecedented speed, accuracy, and optimisation capabilities, enabling engineers to tackle complex, interconnected problems more efficiently than before. This AI-driven approach provides a clear path for engineers to harness dormant data, streamline workflows, and achieve superior designs faster.

Jacomo Corbo, CEO and Co-Founder of PhysicsX