During the historic Apollo missions on the Moon, astronauts were usually equipped with a small book containing checklists and instructions for every action and event. These paper tips proved to be very useful, but still required astronauts to rely on their memories to access critical information. Taking these drawbacks and huge danger of this system into consideration, space agencies decided to use virtual interfaces to expand human possibilities in space and save lives.
An early form of virtual reality (VR) was invented in 1957 by Morton Heilig – a pioneer in virtual reality technology and a filmmaker. His invention, an interactive multimedia theatre known as Sensorama allowed the viewer to watch using stereoscopic images, which let them view the scene from different angles. The first head-mounted display that better matches today’s versions was created by engineers of Philco Corporation and used for training helicopter pilots. Today, virtual reality systems involve wearing a headset that allows a user to see a virtual world and handgear like cockpits that allow them to manipulate objects in the virtual reality. On the other hand, augmented reality (AR) doesn’t move the user to another world. It allows them to see the real world with virtual objects superimposed on it or a composited with the real world.
Unfortunately, some people still see VR and AR as technology invented for gamers and shows, but investors predict that the technology will be used in every industry. The US government space agency has been developing and using VR technology since the 1980s. Private space companies and agencies, like ESA, NASA and CNSA, want to help astronauts recover from the unpredictable and extreme conditions in open space by replacing the basic paper support systems with virtual media interfaces. Based on AR and VR, these solutions could help train astronauts on Earth, see the Martian terrain through the eyes of the Mars Rover, improve space suit designs to provide more information about their health and come up with better designs for space stations.
We will take a look at the trends, statistics and potential of virtual technology for following decades and see if it really can be a profitable venture.
The VR and AR World Market
Today, companies and manufacturers use computer-aided design software. In the future, designers and engineers could totally replace 3D printed physical models during the manufacturing process. Hardware startups are developing more affordable and user-friendly AR and VR headsets. They are also developing new use cases far beyond design and gaming. The technology of an immersive world is being applied to everything from marketing and advertising to medicine and healthcare, and even to military training and space exploration.
Recently published statistics from Nielsen show that only 51% of people worldwide know what a VR or AR is. A year ago, this was only 28%, but it has a tendency to double every year. An emerging factor of such viral interest is the increased number of smartphones with built-in VR functionality. Many mobile apps now allow you to interact with AR content. We can play AR games that put virtual zombies in your living room, let you scan meals at restaurants to determine the ingredients, and shop online by looking at photo-realistic, life-size images of the products. Additionally, it forces the adoption of virtual technologies into industries like healthcare, e-commerce, retail, and design.
The era of VR devices is generating a lot of interests toward virtual tourism, remote event participation, and virtual social interactions that lead to further uses of virtual reality technology. According to TechCrunch, in 2017 AR and VR startups have raised a total of $2.1 billion worth of investments. CCS Insight reported that AR and VR device sales will reach $1.8 billion in 2018, which is equivalent to more than 20 million devices sold.
Samsung, with Galaxy Gear, Microsoft with HoloLens, and HTC with Vive Focus are still the leaders in producing VR gear for the mass market. Apple and Google took the niche of mass consumption AR products and left the corporate market for Microsoft. CCS notes that corporate and consumer AR&VR solutions will develop independently and there is expected to be around 1 million business units sold by 2022 and around 5 million consumers devices.
Thanks to Facebook Spaces, a platform that provides a VR social-media connection between users, AR and VR will have deep integration with technology in social interactions. With haptic technology, it even gives advanced physical interaction and will allow users to touch each other thousands of miles away. This could make large distances less significant, even across continents, or for astronauts in orbit that cannot see their relatives. The goal is to scale up the use of virtual technology and go beyond mass market applications.
Here are 13 industries going straight for AR/VR transformation: marketing and advertising, law enforcement, recruiting, HR, manufacturing and logistics, health and medicine, journalism and media dissemination, film and entertainment, construction and real estate, and space exploration. Due to Perkins Coie research, in the next 12 months, AR and VR are expected to attract the most investment in the gaming, education, and healthcare and medical devices industries.
These potential advantages carry with them yet unsolved technical problems – most notably the cumbersome wires. Wi-Fi and Bluetooth speed is not fast enough to transfer 3D-images in high resolution from the computer to the VR-helmet, so you have to use cables. Secondly, video cards are not yet fast enough to render high the high-resolution images. Because of this, there are delays. There are also delays when moving the head, and some images end up blurred. Developers still have plenty of issues to resolve.
Walking through space. VR and AR for astronaut training
The National Aeronautics and Space Administration (NASA) developed many of the basic concepts of VR technology over the last 40 years of space exploration missions. NASA’s Ames Research Center began to work on early prototypes of VR goggles in the mid-1980s and this first invention has played a significant role in training and guiding astronauts. The main idea of such systems is to bring a high measure of self-sufficiency to crew members working in deep space, where assistance from home may not be immediately available. AR is highly useful for providing users with important information (sensor data, inventory information, repairing instructions) that is usually received through a communication centre or paper instructions.
Why is it so important? For example, the International Space Station has no communication delay. If something happens, scientists working there can immediately send an SOS or ask a question. On Mars, there is a 40-minute delay in communication. So if a problem appears at or near Mars and astronauts have a critical event, they need to wait 40 minutes for an answer.
Travelling to Mars will come with its own set of difficulties, all of which are difficult to recreate exactly on Earth. There are lots of physical health problems, from making sure astronauts muscle mass doesn’t decrease too much to limiting exposure to radiation. While all of these factors could impact the mental health of the crew, the main factor that is directly tied to the wellbeing of astronauts is isolation and confinement. Arctic research centres or submarines can help test people behaviour in closed environments and prepare them for long-haul space travel.
The UK’s National Health Service is developing VR therapy as a way to treat mental health problems, and diagnose and allow astronauts to interact with doctors on earth. There could be huge opportunities for VR to help calm people and assist them during long journeys. A 2006 a study of the American Psychological Association found that pairing exercises with VR headset produced calming effects. VR technology could be used as a form of stress management from diagnosing and treating anxiety in dealing with post-traumatic stress disorder.
The best-known experiment researching human behaviour in closed environments is taking place on Earth and is called NASA’s Human Exploration Research Analog (HERA). HERA is an artificial living habitat that recreates human interactions similar to those they will be met on another planet or in a spaceship. NASA is augmenting this environment with VR and AV training that could give astronauts on their way to another celestial body a break from isolation and the monotony of space. Researchers said that within the HERA project several experiments can be conducted: behavioural health and performance assessments, communication and autonomy studies, human factors evaluations, and medical capability assessments.
Larry Dungan, a researcher at NASA’s Ames Research Center in Moffett Field, presented a hybrid system where people on the ground interact with real-life objects that match a simulation of the International Space Station – for example, grabbing a real-life instrument that is replicated in HTC Vive VR system and moving it in a virtual environment. Larry Dungan, also a coordinator at NASA’s Active Response Gravity Offload System (ARGOS), which uses a lift system to replicate the gravity in different space environments. Dungan said that by combining VR and ARGOS astronauts have the chance to train to immerse themselves in an accurate environment and gravity.
Disrupting space industry with design and science VR
Space companies like Boeing, Sierra Nevada and Lockheed Martin are already using augmented reality in the design and construction of different elements of their spacecraft. Lockheed Martin engineers, for instance, are working on Orion’s Forward Bay Cover (which protects the craft’s parachutes). Engineers using AR work off of instructions to the assembly of the component in augmented reality world, lowering error rates and saving time compared to physical prototyping processes. Lockheed is also working on an AR-based assistant (MAIA) that will combine artificial intelligence with augmented and virtual reality to provide a full digital model of spacecraft, updated in real time. Such assistance will allow a crew to put on an AR visor and see real-time data overlaid on top of physical space.
Project OnSight, presented last year at Kennedy Space Center, gives a clear understanding of the advantages of collaboration in virtual environments. Project OnSight is a virtual reconstruction of the Martian surface, where researchers can work together to set courses for the Mars Rover. Geologist taking part in the project were twice as accurate at defining distances and angles when they were in VR headset. Scientists first used virtual reality realizing that they could run up hills, look around and think in a more agile way. But it can give an even deeper virtual experience with a combination of a headset, haptic gloves and motion trackers. This project has already been implemented by NASA in the Jet Propulsion Laboratory, USA.
Before sending people to space, the next big thing is to bring space to people. Humans have been able to get close to a Delta IV rocket launch that takes off in the VR app created by United Launch Alliance (ULA). Everyone can watch this kind of life events in VR applications, interact, create new experiences or adapt any virtual scene. These kinds of VR tours can be part of the future STEM (science, technology, engineering and math) education and professional training. Besides, virtual reality astronaut training is the most sophisticated way of using VR technology. Today space agencies train astronauts in the aspects of spacewalks (EVA). It’s a manoeuvre in which the astronaut leaves the spaceship and performs some actions outside, such as replacing a camera or antenna.
NASA’s Jet Propulsion Laboratory in Pasadena, California, collaborated with Google to produce the Access Mars project. This immersive VR application provides tours on Martian terrain and shows dunes and valleys explored earlier by NASA’s Curiosity Rover. Katie Stack Morgan, a JPL scientist on the MSL mission, explains that the goal is to take scientists to Mars every single day. Access to Mars was created using data collected by the Mars Rover. It brings otherworldly possibilities down to Earth.
As a conclusion, in next 5 to 7 years, VR will help the public connect with the obscurity of space travel and will likely become a learning tool for students who study astronomy, physics and engineering. Beyond practical uses for training and engineering VR offers a forceful method to share the work we’ve been doing to design efficient space missions and to inspire the next generation of space pioneers.