Envisioning the Naval Metaverse The Indian Navy’s journey towards a Digital Future - Digital Transformation | Virtual Reality | Augmented Reality

Envisioning the Naval Metaverse

The Indian Navy’s Journey towards a Digital Future

Introduction

Digitization is widely accepted to be a game changer for the Navy’s acquisition and operations of its assets. However, end-to-end digitization is still far off. In this paper, we present the vision for a Digital Future that we call the “Naval Metaverse”. The idea is to provide a unified digital environment where the Naval personnel can build, train, operate, and maintain all assets by applying the latest technology for serious applications and practical operations. The naval metaverse will break data silos and provide unprecedented efficiencies in terms of availability of data, including design, operations & maintenance, for naval vessels as well as enable the development of applications that leverage this data. We talk about the overall architecture of the naval metaverse with key building blocks, its benefits, and the challenges that lie ahead. We finally conclude with the current state of Metaverse initiatives in the Indian Navy and future steps from here.

Important Note: Just because of the use of the word “Metaverse”, readers should not assume that this requires the internet. Or it is meant for social networking and entertainment purposes. We envision the Naval Metaverse to be a “secure in-house Metaverse” that’s residing within the Navy’s private network and not exposed to the outside world. Also, the Metaverse does not necessarily mean the use of Virtual Reality (VR), though we envision VR and similar technologies like Augmented and Mixed Reality, to be a big enabler in providing collaborative immersive experiences for various digital use cases.

Motivation & Background

Before we dive into the Metaverse, we provide some motivations behind this work. We first highlight some of the inefficiencies that are screaming for attention.

New vessel production & acquisition for the Navy is a time and capital-intensive activity. However, due to the various complexities involved, these projects are known to suffer from time delays and cost escalations. Shipyards undergo a huge amount of rework due to various inefficiencies in the design to approval to construction cycle. Many of these are due to technology barriers and data silos that limit effective communication between the various stakeholders. Eliminating these inefficiencies is mandatory for the Navy to ramp up its fleet to meet its objectives and mission preparedness.

With fast technological advancements, keeping naval personnel up-to-date with rapidly evolving technologies and weapon systems is a challenge for the Navy. Further, due to limited resources, including training infrastructure and equipment, the quality and quantity of training provided can be severely affected. This coupled with the fact that personnel are transferred every 3 to 5 years, necessitates that the Navy comes up with cost-effective, efficient, and flexible tools to train its personnel.

What is the Metaverse?

As part of ChatGPT, the term “metaverse” refers to “a virtual or digital universe where users can interact with each other and digital environments, often through the use of augmented reality (AR), virtual reality (VR), or other immersive technologies”. Commonly accepted applications of the Metaverse include gaming, social interaction, education, and more.

If we break the above definition, the key underlying technologies in the metaverse include the following:

Digital environments provide a 3D representation of the physical environment including the world around us as well as the people (called Avatars) and objects in the world. The source of this 3D data can include digital terrain models, 3D scans & models of buildings, and as well as 2D information associated with these 3D models. As we speak, there is a massive global effort towards building these 3D environments that provide digital equivalents of real-world assets. There is a substantial effort towards building tools that simplify the process of creating these environments as well as standardization of data formats used to store digital data.

The Naval Metaverse

The Naval Metaverse simply takes the above concepts and maps them to the Naval environment and requirements. However, before looking at the architecture of the Naval Metaverse, we should look at the key benefits that such a platform can bring to the Indian Navy.

Benefits & Use Cases

Though originally created for consumer markets and use cases, the concept of the metaverse holds several key attributes that make it attractive for the Navy. These attributes can revolutionize how the Navy operates, collaborates, and innovates. Here are some of the key attributes:

Collaboration: The metaverse allows for immersive and real-time collaboration between the Navy, Naval shipyards, and other stakeholders across geographies. This has already proven to bring efficiencies to ship design, construction & inspections. The same can be extended to remote support for equipment operations & maintenance.

The above figure shows an illustration of collaboration between the design team at the shipyard and with operations team to validate the ergonomics of an HVAC room on a vessel. Note that the digital environment here is created by the 3D CAD model available with the design teams at the shipyard. The collaborators join from virtual reality or desktop computers, as the case may be. The actual operator who is testing human factors needs to be VR though to be able to do a 1:1 scale navigation and simulation of operations within the vessel.

Training: The metaverse is the ideal platform for hands-on digital training through interactive and gamified experiences. VR-based training is becoming the de-facto standard for training on safety, equipment operations, and maintenance. It’s been proven to save time & cost while improving training efficiency and knowledge retention.

The following figure shows an illustration of an officer training on equipment. The training happens in step-by-step guided mode as well as unguided evaluation mode. With the Metaverse, the training can be done in collaborative mode with multiple trainees in the same virtual environment. Further, since modern VR headsets are compact and low-cost, they can be deployed onboard Naval vessels to train new crew members.

Maintenance: With the use of phygital experiences (physical + digital), the metaverse will replace age-old ways of paper-based or more recently, computer-based documentation (IETMs) with interactive, step-by-step, in-situ manuals that substantially reduce chances of human error and need for experts during maintenance.

The following picture shows an illustration of a maintenance manual created using augmented reality technology. This is like having a machine expert next to the operator giving verbal instructions on each step as they are performed. The fact that the instructions and necessary information are available at the place where the activity has to be performed, eliminates the reliance on memory and hence the potential for errors.

Data and Analytics: The metaverse generates vast amounts of data, allowing the Navy to gather insights, conduct research, and make data-driven decisions. These analytics can be a game changer in terms of ensuring mission readiness, planning maintenance activities as well as monitoring production & acquisitions.

Digital Twins: Longer term, the Navy can create digital twins of its assets, processes, and environments within the metaverse, enabling better monitoring, analysis, and optimization of operations. This requires integrating multiple data sources and using the Metaverse as a ‘window into the Digital Twin’.

The uniqueness of the Naval Metaverse is that it is a “single platform” that allows all of the above use cases. Having a single platform breaks the data silos that traditionally exist between technology components. This will eliminate the constant “recreation” of data thereby resulting in saving huge time & costs for the Navy. For example, currently, the DND and various shipyards create 3D design models of naval vessels using CAD software like Aveva Marine and Foran. However, the same models, if made available as part of the Naval Metaverse, will allow leveraging the same data for training the crew as well as ongoing operations & maintenance. It’s also important to note that the use cases of the Naval Metaverse will keep evolving as the technology landscape continues to evolve. With the advent of Artificial Intelligence and allied areas, we are simply limited by our own imagination of the possibilities.

Architecture

If we draw parallels with the definition of the Metaverse from the previous section, we can easily see a similar architecture evolve for the Naval Metaverse. Note that this is NOT the only architecture and many variants of the same are possible. However, we believe that the high-level structure will be similar and the following can be extended to encompass additional use cases.

The use of VR, AR, or other digital devices like tablets is driven by specific applications or use cases that the Naval Metaverse provides to its users. It’s easy to see the use of VR for Training and AR for Construction use cases. However, the use of immersive technologies is not mandatory for all applications. Note that the above list is indicative and a comprehensive list of applications can run into hundreds.

The Digital Environment is represented by a centralized metaverse database. This is a digital representation of the environment that the Navy operates in, the core component being the information related to the Navy’s assets.
- For newer vessels, fortunately, the 3D design data is available with the Indian DPSUs and DND. These can directly be integrated using automatic no-loss data translation tools already in place with the DND and DPSUs. Older vessels need to be digitized using reality capture technology including 3D scanning and modeling activity. Luckily, this technology has also evolved substantially over the recent past.
- In addition to 3D content, there are multiple IT software used onboard as well as off the vessel for specific use cases. To derive maximum return on investment, these systems need to be integrated with the Metaverse environment either using a one-time data migration or using API-level real-time integrations. These integrations can be done in a phased manner as new applications & use cases demand them. For example, AR-based IETMs can be built by offline migration of CAD data from design teams. However, the IOT-based Digital Twin application would require real-time API integration.
The interactions between various users will be over a secure communication channel. Since Indian defense does not allow internet, all of this will be available on the Navy’s internal network. Regardless, all communication needs to be encrypted and user authentication to be done using standards created by the Navy.

Challenges

It’s important to note that while the Naval metaverse offers exciting possibilities, its development and adoption are ongoing, and there are still challenges and considerations that the Navy and technology developers need to address as they explore these opportunities:

Security and Privacy: Ensuring the security and privacy of data and interactions within the metaverse is crucial for the Navy’s use cases. Robust cybersecurity measures are essential but also the Navy needs to have a clear-cut policy to enable building and deploying such applications. In addition to the use of the Navy’s private network, the server platform needs to be certified for strict security standards like SOC 2 Type 2. In addition, SSO integration with Naval systems should be possible for large-scale deployments.

Interoperability: Standards for interoperability between different metaverse applications and technologies are important to ensure seamless integration into existing Naval systems. For the centralized database, we recommend the use of Industry standard GLTF file format for storing 3D assets along with suitable data conversion utilities that enable quickly adding digital assets to the environment. Also, the provision of well-documented APIs that allow future applications to be able to work with the metaverse environment. Likewise, the platform should enable the same content created for VR applications to be used in AR environments.

Data sizes and fidelity: One key difference between the Naval Metaverse and consumer Metaverse is the size of the data. Due to the complexity of Naval assets, the Metaverse applications should be able to handle extremely large amounts of 3D data using sophisticated data management techniques. Further, there should be no data loss during the migration process. For example, in order to build a maintenance application, the migration from CAD data should preserve metadata enabling users to get relevant information about materials and OEM information.

Digitization of Assets: There is no shortcut to digitizing assets for which 3D design data is not available with the Navy. Fortunately, such a massive effort typically needs to be done in a phased manner. We feel that the digitization of assets can be taken up by the Dockyards along with regular maintenance and refit exercises. 3D scanning technology has evolved substantially and in conjunction with modern software, the process of creating digital assets can be done in a relatively short amount of time and cost.

Current Status

The seeds for the Naval Metaverse were sown in 2019, with the Directorate of Naval Design (DND) setting up its Virtual Reality center which collaborates with the Virtual Reality center in Mazagon Dock Shipbuilders Limited (MDL). This was the first step towards enabling virtual collaboration between the Navy and the leading Naval shipyard to streamline design reviews and approvals. Similar setups have been done by other shipyards including Garden Reach Shipbuilders & Engineers (GRSE), Larsen & Toubro Defence (L&T), and more recently, Goa Shipyard Limited (GSL). The technology itself has enabled the streamlining of traditional processes like physical line-out inspections (LOIs), which were time-consuming and oftentimes inaccurate. The use of Digital technologies has led to improvements in efficiencies, reduction in rework during ship construction, improved stakeholder engagement during design reviews & elimination of physical prototypes & mockups.

The Digital Inspections project spearheaded by WOT (Mbi) at MDL was another step in this direction that enabled naval inspectors to leverage 3D design data onboard the vessels under construction. This has transformed the inspection process which continues to evolve after 1.5 years of successful operations.

Currently, all the inspection checklists are available on an Apple iPad along with 3D design models and partlist from Aveva Marine. This has eliminated the paper-based process along with streamlining the creation of documentation. The management dashboards help senior officers track the progress of inspections and identify & resolve bottlenecks as they arise.

In addition to the above, there are various VR training, operations & maintenance initiatives across the Navy. However, most of them are being developed in silos and do not leverage a scalable no-code metaverse platform that addresses the challenges mentioned in the previous section. This would limit their usage & deployment across the Navy.

Exxar technology, represented by VizExperts in India, provides an indigenous solution that can address the requirements of the Naval Metaverse holistically. With its no-code content authoring interface that allows loading CAD models without any data loss, the platform enables developing applications without dependency on the OEM for content creation. It has been tested inside the Naval shipyards for a few years for design reviews, collaboration, and digital inspections and is currently being evaluated for ship production activities. The technology is already being used by global engineering firms like Baker Hughes, Tesla, AtlasCopco, Larsen & Toubro & Scania along with Indian PSUs like NTPC, MDL, GRSE, and GSL.

Future Work

Based on past success, we are quite optimistic about the future of this technology in the Navy. From a deployment point of view, the following figure shows the technology’s progression from a use case perspective, moving from design collaboration all the way to addressing Digital Twin requirements.

We recommend the following steps as a phased-wise approach to building the Naval Metaverse. Note that these need not be built sequentially and multiple stages can potentially be executed in parallel:

As a first step, ensure that 3D design models for ships built within India are made available to the Navy. In the future, this can be made a mandatory requirement of the ship commissioning process in addition to the handover of other documentation and drawings. The Navy can setup a secure Metaverse server which can be a single repository for housing these data.
The next logical step is to execute pilot projects for building VR-based training content and AR-based maintenance manuals for vessels whose 3D design data is available. Starting with critical operations, the training content can evolve over a period of time and various vessels can have their own internal training setups in addition to the Navy’s dedicated training institutes. Likewise, the Navy can choose to initiate pilots for AR-enabled IETMs by either migrating existing IETMs or initiating this for to-be-commissioned vessels directly at the shipyard level.
The third step will be to build more sophisticated real-time digital twin applications that are powered by the Metaverse through integrations with other software solutions that maintain asset management or live sensor data. Building an AI-enabled analytics engine on top of these applications will enable the Metaverse to enable smart decision-making for use cases like predictive maintenance, autonomous operations, and others.

Conclusion

The Naval Metaverse promises to provide a digital platform for the Navy to address multiple challenges and bring in effectiveness and efficiencies in areas of concern. This requires ensuring that multiple data silos be eliminated and a single digital view or environment of the physical assets be created. Focused applications can then be built that leverage this digital environment to address specific use cases like training, maintenance, and operations to deliver the perceived benefits.

Fortunately, this indigenous technology has already been proven in the ship design, inspections and production areas at Indian Naval shipyards. With proper initiatives and management support, the same technology is now poised to be adapted for additional downstream use cases. A phased approach for building the Naval Metaverse is also proposed that includes ideas for pilot projects to test the technology at a small scale before scaling it to additional users and use cases.