The technologies of Virtual and Augmented Realities, collectively known as XR Realities could hold the key to providing the added value perceived to be ‘missing’ from standard video communication. These new technologies can make digital communications more immersive, information-rich and persuasive than a simple video image and audio call. At the very highest level. XR technologies can create a total environment that collaborators can share and use as a platform for interaction. In a week that would usually see the AV News team in Amsterdam for VR Days we assess the state-of-the-art.
It has been a few years since we started to attend about Virtual and Augmented Realities, and in that time we have seen a variety of new technologies emerge and claim a place in the market. And yet event after event sees us presented with projects, proofs of concept and trials. What we are yet to see are commercial solutions and products available to buy.
Before we consider why this might be, let’s review our expectations. Augmented reality (AR) is an interactive experience in a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory.
AR can be defined as a system that fulfils three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive (i.e. additive to the natural environment), or destructive (i.e. masking of the natural environment). This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one’s ongoing perception of a real-world environment, whereas virtual reality completely replaces the user’s real-world environment with a simulated one. Augmented reality is related to two largely synonymous terms: mixed reality and computer-mediated reality.
In a collaborative interaction, perhaps we might consider a development project in which an architect’s schematic is augmented with material specification or budgetary information. This would provide collaborators with immediate feedback on the costs of any changes. [One can’t help wondering how much time and money this might have saved when the late Zaha Hadid was presenting the designs for the 2012 Olympics to the then government minister Tessa Jowell?]
An enhanced and interactive computer model with additional and useful information integrated should a relatively straightforward solution to sell – provided the cost is reasonable. So what solutions are available?
AR tools and solutions
AR is still young, with many new types of technology still emerging. Some of the main types of AR software on the market at the moment include:
- AR visualization software – This type of software enables organizations to create immersive experiences for consumers to interact with. AR visualization software users can upload 3D content and scale the image, adjust the colour, and incorporate the additional details needed .
- AR content management system (CMS) – An AR CMS lets users bulk upload raw 3D content that will eventually become the basis for AR experiences. This content can be managed and edited within the platform.
- AR SDK – These tools allow users to build digital objects that will blend into the real world that will eventually become fully fledged AR experiences.
- AR WYSIWYG editor software – This software enables users with limited to no coding background to create customised AR experiences. These tools have drag-and-drop capabilities that let users upload 3D objects and drop them directly into previously designed scenes.
- AR game engine software – These solutions give game developers the framework for creating AR video game experiences. Using AR game engine software, users can create and edit 3D characters that can interact with the real world.
- AR training simulator software – AR training simulator software leverages AR technology to train employees for certain jobs.
- Industrial AR platforms – These solutions are typically used by organisations in the industrial field. These tools include interactive AR content that improves these employees’ productivity, effectiveness, and safety.
Examples of highly rated AR development software include: ARCore; ZapWorks; and Vuforia Engine. ARGame Engine software titles include:Unity; Unreal Engine; and HeroMirror. Finally, AR training simulator software products include: Amazon Sumerian and MaestroAR (although the latter is not yet rated).
Software costs aside, one of the biggest factors that has hindered AR from becoming mainstream is the cost. It can be very expensive to purchase the hardware to support AR technology. Streaming the content can also very costly. Content for these solutions needs to be streamed in a very high resolution and rendered at a high refresh rate. This content also requires a large bandwidth for streaming. All these factors add up, making potential users wary of adopting AR.
Business case
Of course, the cost comparison with an AR project can often be expressed in terms of the end result. For example, let’s take a retail example: users in the retail industry can leverage AR technology so consumers can virtually test out products before they make a purchase. AR retail applications allow users to upload a photo of themselves and visualise what a particular piece of clothing would look like on their body. Shoppers could also use these kinds of applications to visualise what a piece of furniture would look like in their house. Both examples here would offset the costs of the AR solution against the likely costs of customer returns of rejected goods.
Other potential users of AR technology are students and school pupils. If a teacher was conducting a lesson on astronomy, AR software could project a map of the solar system so students could visualise the subject in an exciting and immersive way. Later in life, employees performing manual labour can wear AR glasses to help with repair and maintenance jobs. AR software can be used to project valuable data and guide the user in the task. At the other extreme, surgeons, can use AR technology for training purposes. AR technology can help trainee surgeons visualise what the actual act of surgery would be like.
Virtual worlds
There is a growing consensus around the value of Augmented Reality but taking up a level to the creation of complete virtual environments seems to prompt talk about science fiction, Holodecks and the like. Why should this be? While consumer level VR is concerned with gaming and headsets, professional applications are largely concerned with Virtual Reality Caves and simulators. A VR ‘cave automatic virtual environment’ (better known by the recursive acronym CAVE) is an immersive virtual reality environment where projectors are directed to between three and six of the walls of a room-sized cube.
One of the companies in the high-end VR field that AV News follows regularly is ST Engineering Antycip. One of the company’s recent CAVE projects saw The University of Huddersfield install a state-of-the-art VR CAVE system at its new Barbara Hepworth Building, named after the famous sculptor. Home of the School of Art, Design and Architecture, the building boasts a 7,500 sqm space and is designed to support multi-disciplinary learning. The VR CAVE, housed in the School’s Phidias Lab, was installed by VR integration specialist ST Engineering Antycip (formerly Antycip Simulation) on behalf of its client Roche AV.
Designed to function as an experimental AR and head-mounted display (HMD) facility, the Phidias Lab promotes the development of the school’s digital strategy through encouraging the creation of digital content and immersive VR. It is home to an eight-screen video wall with several modes for exhibiting and presenting, a motion capture area, a feature sound and lighting system and now, a highly advanced immersive CAVE system.
Surrounded by a 16-camera Vicon motion capture system, the CAVE supports a wide range of virtual environment needs. It provides students from a variety of academic subjects the opportunity to develop their design concepts, without the need for potentially expensive physical models or printed material. Antycip was tasked with providing a software platform that uniquely offers the ability to instantaneously bring industry standard applications into a stereoscopic VR environment, saving engineering time and effort while providing a fully interactive and unforgettable virtual experience.
Antycip designed a three-face CAVE, comprising of three solid state Barco F90 laser projectors. Boasting 4K resolution and 3D capability, the projectors are equipped with ultra-short throw lens optics to address the primary front projection face of the CAVE and the wing face positioned on the right side. Both projectors are inverted and enable huge savings on the real estate required for the rear projection throw distance behind the solid specialist rigid acrylic substrate screens. A third F90 projector utilises a specialist mirror to fold the light path down to the floor, where the content is combined with the two rear projection surfaces, thus offering a truly immersive display environment. Each projector is fed a high bandwidth stereoscopic signal using pure fibre optic based cable links to avoid potential electromagnetic interference.
Vicon spatial tracking cameras are calibrated to offer an interactive volume within the display, so that a dynamic eyepoint can be tracked for a single user wearing the supplied RF synchronised active shutter glasses. This tracking system also supports the interactions from a wireless handheld device, used for navigation within the virtual content as well as to initiate and control any interactive feature activated from the TechViz software while immersed. The CAVE design enables unique folded lens optics to maximise the room’s real estate while offering vibrant high-resolution images to this faceted array. The solution is driven by dedicated PC-image generators with GPUs synchronised in both hardware and software to address the demands of rendering large complex datasets.
Due to the varied use of the CAVE, the university required a system which could be used by students from a range of technical backgrounds and abilities. “We needed to mitigate the need for complicated programming or the use of additional programmes before a model could be viewed in VR,” said Stephen Calcutt, technical manager and co-ordinator for the University of Huddersfield. “ST Engineering Antycip recommended that we use the TechViz software as its support of native applications such as Solidworks, Navisworks, Revit and Rhino, which are used by the School of Art, Design and Architecture, makes it a vital component of the ease of use system. It allows students to create and instantly display 3D stereoscopic interactive content within the CAVE that is dynamically linked to the applications they know, without having to convert complicated data first.”
Training potential
Related to education applications, high-end VR has found a growing niche in training. Bau Bildung Sachsen (BBS), the competence centre for construction machine technology in the Free State of Saxony (Germany), has taken delivery of six of CM Labs’ Vortex Edge Max simulators to help train the next generation of construction workers in the region.
The simulators will be used at BBS’s Glauchau location, where they will help facilitate initial vocational training in over 20 trades in the construction industry, including exercises to support fine motor skills, endurance, and accuracy when using heavy machinery. The deal came about thanks to a three-way partnership between ST Engineering Antycip, CM Labs, and KRS Solutions.
The simulator comes bundled with a full fleet of cranes, CM Labs’ complete earthmoving catalogue, or both, for a total of 10 machines in one unit. BBS decided that the full package would be ideal for its broad offering, making the simulator’s versatility one of the key factors in choosing the Vortex Edge Max.
“The benefits of using virtual training methods are myriad,” said Enrico Salcuni, sales manager for Italy-DACH, ST Engineering Antycip. “They provide a secure and granted return on investment, as well as allowing operators to train in a safe and hazard free environment. These systems are becoming more and more common among companies, schools, and training centres alike, and I would suggest that everyone try it first-hand.”
The Vortex Edge Max simulators were delivered to BBS fully embedded with CM Labs’ Smart Training Technology and a motion platform that replicates the feel of the real equipment, as well as best-in-class visuals and exercises that replicate work site conditions.
“The graphics, motion feedback, and dynamics are all so well defined and accurate that it really feels as if the trainees are operating a real machine,” continued SalcuniS. “This method can really help institutions to save money on equipment, fuel, and maintenance, as well as enabling you to make mistakes that, in the real world, would be dangerous or even catastrophic. Trainees can learn from these mistakes, improve, and return to their families safe and sound; for this reason alone, the Vortex Edge Max is invaluable.”
Cost benefits
Of course, the costs of CAVE and other high-end VR solutions can be daunting, but the value in VR solutions, for research and development, is reflected both in the savings they represent when compared to physical prototyping, and in the benefits they bring with greaterflexibility earlier in the design process.Virtual reality is more than just another research method: “Virtual reality allows you to maximise the success of your design projects by continually reviewing and modifying them before manufacture and launch. It helps inform marketing strategies and campaigns, “says John Mould of Antycip.
One of the companies in the high-end VR field that AV News follows regularly is ST Engineering Antycip. One of the company’s recent CAVE projects saw The University of Huddersfield install a state-of-the-art VR CAVE system at its new Barbara Hepworth Building.
While consumer level VR is concerned with gaming and headsets, professional applications are largely concerned with Virtual Reality Caves and simulators.
