Comment: Moving HPC to the cloud

With a few exceptions, say areas of national defence or critical national infrastructure (CNI), yes, HPC can be in the Cloud. But there’s much to consider, such as cost – long or short term, who maintains and manages it, and the available skills, and time, to oversee demands.

Furthermore, commercial decisions need factoring given the investments required and, security is always a priority – as research is often conducted on behalf of governments, big pharma, finance, and even high profile and high-ticket sports such as Formula 1.

None of this should preclude cloud-based HPC bringing considerable power and potential to these situations.

According to research report by Market Research Future (MRFR), in April 2023, the cloud HPC market size was valued $5.5bn in 2022 and is predicted to grow over the next seven years at a compound annual growth rate (CAGR) of approximately 16.68 per cent, to reach $16.19bn by the end of the decade. That’s big business, and it’s where commercial gains and intellectual property meet.

Total cost of ownership

Depending upon where your influence lies in an organisation, your objective may be swayed by different factors. A CFO may focus upon cost (especially Capex) while a CTO or CEO might see investment, or total cost of ownership (TCO).

It’s true, with an on-premise infrastructure, organisations have complete control of all operations and will own or at least manage and maintain both the server and datacentre environment.

However, as all CFOs will testify, this is a fixed and depreciating asset, which needs to be written down over a period of time; usually five years. However, capital outlay for hardware and infrastructure typically accounts for only a third of the overall costs of running an HPC environment. The remaining two thirds are eaten up in maintenance and running costs, and may fall within an operations budget.

Research and higher education organisations must be able to demonstrate ROI, which of course is only right. However, the timescale in which you measure it is as important to understand, as is the TCO.

Security Matters

Security, particularly around data breaches and fending of cyber-attacks, remains one of the most urgent matters of concern among IT administrators in any field of operations. But in academia and research it has particular potency when it comes to cloud computing.

Last year, the average global cost of a successful cyber-attack reached a record high of $4.35m (£3.41m), as reported in IBM's 2022 Data Breach Report. Until there is a wider understanding of how to overcome cyber-attacks, and the demands that come with them, security issues will remain the most urgent matters of concern among IT administrators for cloud computing.

If researchers in fields like genomics remain unaware of the potential to leverage the computational capacity and data accessibility provided by cloud services, the progress of research will potentially be hampered. With the appropriate support in place and the right experience of cloud based HPC behind them, there is every opportunity to unlock the full power of the cloud securely.

Minding the skills gap

In many organisations that require HPC, there exists the ‘workforce development / workforce management’ dilemma. In essence, organisations realise they there is a skill and resource shortage, but continue to have reservation around outsourcing. Meanwhile, many good research staff members in higher education make the move to corporate, private sector, where there is more funding and reward.

The challenge, and it’s one we are working closely with customers on, is how to retain a level of in-house knowledge, while ensuring the outsourced support is also there. Over time, needs will evolve and the nature of that support may also evolve, because managing HPC is not the same as managing enterprise IT.

Crucially, universities and research departments need complete peace of mind in the transition to the cloud once they realise it is the best option for them. If appropriately outsourced within a robust managed service arrangement, they can draw on the deep-level expertise to steer them seamlessly onto the cloud, or indeed onto a hybrid model. The decision about which environment is best suited will always include cost and security considerations, but neither should be looked at in isolation.

Owen Thomas, co-founder of Red Oak Consulting