XR, AI, 5G, SCT Editor Andy Fawkes and industry experts look at some of the hottest trends in simulation and training technologies.
There are numerous predictions of how, with advances in technology, the world might change over the next decade. Many centre around AI, the Internet, human interfaces, energy supply, health and transport, but how might safety critical sector simulation and training change? Predicting the future can be fraught because human endeavour is often chaotic and what seem like certain trends are consigned to history as other trends take over. However, as it is the start of a new decade, we asked industry experts for their view and have put together a guide to some of the trends in S&T-related technology.
Innovation
One does not need to be in the S&T community to observe that the take up of technologies, particularly consumer-focused, are accelerating. For example, mobile devices have reached levels of worldwide adoption in 10+ years which took telephone landlines 100+ years. Further, technologies once thought of as science fiction can now be found in our homes, such as Amazon’s Alexa AI.
We are seeing both military customers and industry looking to this wider world and seeking the same level of innovation and agility. Sébastien Lozé, Industry Manager Simulations at Epic Games told SCT: “there are three steps of technology innovation: Discovery (where many of these trends are today); Dissemination (where the target audience is reached); and Adoption”. Companies such as CAE are putting their emphasis on “digital innovation”. Their 2019 Annual Report highlights their Project Digital Intelligence innovation program aimed at “cutting edge digital technologies and systems such as big data artificial intelligence, cloud computing, cybersecurity and augmented/virtual reality”.
S&T is not a greenfield site however, with training programmes already contracted out so the challenge for the 2020s will be to incentivise existing players to innovate whilst encouraging and supporting new entrants in what can seem a complex and tough market to break into.
Train Faster, Anywhere, Anytime
Customers are seeking training pipelines that can adapt to changes in demand, not tied to specific locations and more agile to operational demands. Some term this accessibility and flexibility as the “Netflixisation” of training.
Lozé told SCT, “Thanks to advances in cloud services (Google Cloud Services, Microsoft Azure, AWS), 5G technology, real-time virtual environments such as Unreal, and more – training will no longer be locked to one physical location. Instead, advanced simulations will be mobile, and trainees will greatly benefit from the convenience of learning at their own pace, and in their own space (either personal places or workplaces).”
AI, Data and Analytics
AI, Automation, Machine Learning, Big Data, Analytics et al are terms that regularly make the headlines and technology commentators anticipate major advances in these fields over this decade. Inevitably there will be an impact on S&T. Advances in training data capture and analysis could improve our understanding of training outcomes and help tailor training to individuals.
For decades AI has been important to the S&T community to reduce the number of human role players but now it may help create simulations. Lozé said: “today, content creation for simulation is heavily reliant on humans. GEOINT data editors build accurate datasets in a process that is nothing short of time consuming. AI can, and will, significantly reduce the time cost of manual preparation, freeing the GEOINT community to focus more on data analysis than on data maintenance. This is just one example of how content generation will be revolutionized by AI, and we can expect a swell of other improvements.”
As for the AI in simulations, Pete Morrison, Chief Commercial Officer, Bohemia Interactive Simulations, cautioned that: “there is huge excitement around machine learning, but I think that the change will be gradual as there is a tremendous difference between one of Google’s computers playing Go and a military commander making tactical decisions. I think it all comes down to whether the problem is bounded or not and how much data is available. It’s all about the rule sets and then having the data in order to effectively teach the AI to make good decisions.”
XR
MS&T has reported on the significant number of XR-based applications (AR-MR-VR) seen at I/ITSEC 2019. Simulation companies have been exploring the use of XR for some time, particularly since affordable products such Oculus Rift was released in 2013. But there have been challenges for VR. Morrison explained that for VR-based pilot training: “we hit a brick wall in the lack of haptic feedback that is required for pilots to be able to interact with the cockpit.” Morrison did not see the haptic problem being solved fully in the 2020s as: “… there’s no good technology for allowing an operator of any system to interact with the virtual world in a realistic way and part of the problem is that these systems are tremendously complex.”
MR (mixed reality) however, shows considerable promise. The Varjo MR HMD allows the trainee to see and interact with a physical cockpit but seamlessness see the virtual world out of the cockpit. Morrison believes that: “over the next 10 years MR will absolutely crack it as it is a factor of processing power and resolutions will increase and eventually the scene in the headset will be good enough. It might take another few years, but MR will replace traditional dome-based flight simulation.”
Lozé is looking to other XR innovations such as the Open XR initiative. “This will allow for flexible deployment to all methods of XR, allowing simulation creators to focus on the curriculum at hand, and not on the display of a pixel on one HMD or unique display. Choosing your method of XR will soon be as trivial as mono vs stereo vs 5.1 sound, and this will unlock all kinds of new opportunities in the next ten years.”
The British Army is looking beyond individual training to collective/team XR-based training. In 2019 it demonstrated that 30+ soldiers with XR HMDs could operate together in the same virtual world in a variety of training scenarios and in 2020 this XR-based system will be deployed to barracks as a mobile training service. In future such deployable and flexible systems could offer collective training opportunities not easily provided through more traditional fixed training systems.
Improved Simulation
The resolution of displays is ever improving. 8K televisions are on the market and XR displays are heading towards eye resolution. “Advancements in GPU capabilities are ushering in a new era of display techniques (ray tracing, PBR, photogrammetry, virtual humans), ensuring that simulations not only behave realistically but also with a high level of authenticity and believability,” Lozé told SCT. With recruits experiencing high fidelity games and video content “the next generation of trainees demands a more immersive training experience.”
Digital Twins
Although the term “digital twin” has been used for some time it is on the rise with the ever-increasing digitisation of products and activities. A digital twin-based approach offers the prospect of consistent, reusable, and available data in support of the procurement of military systems and platforms and all associated simulations through life, ranging from analysis through to training and mission planning. Indeed, keeping simulators up to date with their corresponding system/platform is an ongoing challenge and a digital twin approach could, in the future, help build understanding between the procurement and S&T communities.
We can apply digital twins to hardware but what about people? UK start-up, AERALIS is developing a modular fighter jet trainer aircraft and looking at novel ways of delivering the syllabus. AERALIS Strategy Director Tim Davies explained that they are looking to streamline and improve the complex and costly process of military flying training by incentivising the training process. They are proposing the “use of a ‘digital twin’ which represents the model pilot that a student needs to emulate, and which the student then tries to match by working towards making their digital record of flying and combat performance equal to that of the twin,” Davies explained. Based on gamification principles, the pilot not only can compare themselves with the digital twin, but they can share their digital record across the class so that students can learn from each other to improve their own performance. In addition, the training service provider will have anonymous data to compare not only student performance but also instructor performance and syllabus design. “For the first time a provider will have quantitative basis on which to develop better training systems in the future,” Davies said.
Another example of digital twins is the Brightline and Teslasuit demonstration at I/ITSEC 2019. Here a trainee can train and rehearse their tasks in a virtual world and then carry out the same tasks wearing a Teslasuit which provides tactile feedback to guide them towards the behaviour of the digital twin they created in the virtual world or that of an exemplar student.
Exemplar digital twins and digital student records combined with the security and transparency of blockchain technology together with links to HR and recruitment systems could provide powerful insights to decision makers, from the students themselves through to the enterprise. However, crossing the many organisational barriers to make this a reality would be a very significant challenge, even over a 10-year period.
Interoperability
The military S&T community has been successful at developing interoperability standards, but can we expect change? Morrison believes: “… that we are actually getting into the end of what I call the traditional interoperability era. HLA/DIS are good standards created for good reasons, widely adopted, and working. But the problem is that the rest of the world outside military simulation doesn’t use HLA/DIS. They use web services and standardised means of communications like Google protocol buffers in order to deal with these very complex systems.” There is also the HLA/DIS skill issue. Morrison continued: “You can leverage the technologies that the broader industry is using, exploiting its skill base and not needing to train an HLA engineer. Over the next 10 years, everybody’s applications are going to move to a much more web-enabled architecture based upon web services or micro services.”
New Business Model
Lozé sees changes in the way S&T are procured in this decade. He calls it “business model evolution” and continued, “it is an evolution of the business model, simulation as a service, a more modular structure of simulators relying on standards. This is paving the way for new entrants in the industry that come from the non-military domain with a different expertise. We are seeing this with HTX Labs for example which is delivering important elements of Pilot Training Next for the USAF. This type of new player is bringing a refreshing tone to the industry and the main prime contractors are displaying a surprisingly agile mindset by teaming up with this new blood. On top of the technology innovation, we can easily foresee a strong business model innovation wave coming.”