Augmented reality (AR) can add value to mining and construction projects in a variety of areas. In this article I present three of my favourite applications and explain the pros/cons of some common AR hardware options.
Mining and construction projects have many similarities. They are spatially complex, busy environments where decisions need to be made quickly to ensure a safe and successful operation. For this to occur, field workers need access to information, planning teams need to conceptualise the project in 4D (a 3D timeline), and management need clear project updates.
A key challenge for mining and construction projects is the ability to visualise the whole operation in 3D. This is needed to allow better comprehension of design and the ability to detect problems before they arise. Some workers find it intuitive to convert a plan or design into a 3D mental model. However, in a collaborative team the individual stakeholders can each build a different picture in their minds, leading to miscommunication.
Utilising AR technology to display a 3D model of the project “on the table” is a great way to ensure that everyone is on the same page. Benefits of this approach include:
1. Collaborate and plan around a 4D model of the project
Using AR as a communication tool for meetings and problem solving is a great way to improve planning efficiency. When information is presented in 3D it is easier to understand, it reduces the risk that an element has been overlooked, and ensures a diverse group of stakeholders are all able to understand the problem and contribute to the resolution.
These 3D models can be animated with the timeline sequence of the project, updated with recent events, and even include live equipment positions. This 4D view helps a team to understand the big picture and highlight spatial conflicts in the schedule before they pose a problem on site. Everyone quickly understands the project and what needs to be done.
2. Bring design information into the field
AR displays 3D computer-generated information in context of the physical world. In complex projects, being able to visualise a design in the field is highly valuable. Imagine assessing progress against a virtual 3D schedule, assessing as-built against design, looking at installation instructions or even looking through barriers to visualise internal structures. These benefits enable workers to do the job quicker, more accurately and with less re-work.
3. Give better boardroom presentations
Time in front of the board is valuable. If a project needs a design variation, an unexpected hazard or a scheduling conflict discussed, then time shouldn’t be wasted painting a picture of the operation. Presenting a detailed 3D model using AR can quickly communicate the problem, help the board understand the options and get to a decision. If you are presenting a new idea or pitching an opportunity, AR can not only communicate the concept quickly, but grab attention and help you stand out to key decision makers.
Animations of recent events and projections into the future can be included into these visualisations. Showcasing project evolution that is easy to understand. An informed board is a happy board, with AR the whole project can be presented, no black boxes.
Common Augmented Reality Devices
AR can be experienced with a variety of hardware, suitable for different applications and budgets. This hardware is generally either headset-based AR or mobile-based AR (using a smartphone or tablet). Each of these systems have a variety of software available to enable AR experiences. However, off-the-shelf AR software can have limitations, so many projects require the development of customised software applications to suit their specific needs.
Most AR devices use a series of on-board cameras to monitor the space around the user and build up a real-time 3D “map” of the environment. The device then uses this “map” to anchor the computer-generated models and visualisations in the physical world, either sitting on a real object or floating in space. This allows you to walk around the space and have these 3D objects stay fixed to a position in the physical world for you to explore.
AR can be experienced via a headset device like the Microsoft Hololens 2 or the hard-hat integrated Trimble XR10 (pictured above). AR headsets display the projected image on lenses in front of your eyes so you can look through the visualisation and still see the physical world around you. This reduces the risk of motion sickness as the spatial perception of the physical world is used to “ground” the user.
The on-board cameras on an AR headset have a second function in that they can also track the user’s hands and recognise hand gestures. This can be used to select virtual menus, grab and move objects, or interact with data. Voice command support also allows the user to interact with the headset and control displayed information completely hands-free if desired.
- Provides a true 3D experience with depth perception provided by the two lenses
- Allows a hands-free experience to use real-world objects (eg, maintenance tools)
- Needs a device per person (multiple people can’t use the same device simultaneously)
- Hard to assist another user with their visualisation as you can’t see what they’re seeing (without streaming their view to an external screen)
A version of AR can be performed via modern mobile devices. As you move around and look through the device’s screen you will see the AR object sitting in-front of you. This presents a perspective 3D view where you can look around the object and interact with it via the touchscreen. Some new mobile devices (like the iPad Pro and iPhone 12) also come with built-in Lidar sensors to make the placement of AR objects faster and more accurate.
This form of AR is very intuitive as you look through your mobile screen as a “window” into that virtual world. This is less immersive than headset based systems, but it is very accessible (most people already have devices capable of delivering mobile AR experiences). An added benefit is that it’s easy to assist people when they are using it as a group of people to all see the same screen and share the visualisation.
AR is making its way into the toolset of workers in a variety of industries. There are many beneficial applications for AR in mining and construction projects, including:
- Improving project planning by enabling collaboration around a 4D model of the project
- Increasing the quality of work in the field by being able to visualise designs in 3D
- Assisting board members to understand project evolution and make better decisions
The benefits of AR can be realised using a variety of hardware, with the most common being AR headsets and mobile-based AR. Once the hardware has been selected the software can be purchased or created to suit the specific needs of the project.
At Vantage Interactive, we specialise in developing custom AR and VR apps for our clients to help them improve their project communication. These apps can be customised with your content upfront or configured so you can update the information as the project evolves. You can see an example of some of our recent AR projects here: https://vantageinteractive.com.au/our-work/augmented-reality-ar
Thanks for reading! Please get in touch if you would like more info or have any questions.