Contained in:
Book Chapter

A Framework for Realistic Virtual Representation for Immersive Training Environments.

  • Caolan Plumb
  • Farzad Pour Rahimian
  • Diptangshu Pandit
  • Hannah Thomas
  • Nigel Clark

As mixed-reality (XR) technology becomes more available, virtually simulated training scenarios have shown great potential in enhancing training effectiveness. Realistic virtual representation plays a crucial role in creating immersive experiences that closely mimic real-world scenarios. With reference to previous methodological developments in the creation of information-rich digital reconstructions, this paper proposes a framework encompassing key components of the 3D scanning pipeline. While 3D scanning techniques have advanced significantly, several challenges persist in the field. These challenges include data acquisition, noise reduction, mesh and texture optimisation, and separation of components for independent interaction. These complexities necessitate the search for an optimised framework that addresses these challenges and provides practical solutions for creating realistic virtual representations in immersive training environments. The following exploration acknowledges and addresses challenges presented by the photogrammetry and laser-scanning pipeline, seeking to prepare scanned assets for real-time virtual simulation in a games-engine. This methodology employs both a camera and handheld laser-scanner for accurate data acquisition. Reality Capture is used to combine the geometric data and surface detail of the equipment. To clean the scanned asset, Blender is used for mesh retopology and reprojection of scanned textures, and attention given to correct lighting details and normal mapping, thus preparing the equipment to be interacted with by Virtual Reality (VR) users within Unreal Engine. By combining these elements, the proposed framework enables realistic representation of industrial equipment for the creation of training scenarios that closely resemble real-world contexts

  • Keywords:
  • Digital twin; 3D reconstruction; Virtual reality; Laser scanning; Photogrammetry; Training simulation; Unreal Engine
+ Show More

Caolan Plumb

Teesside University, United Kingdom

Farzad Pour Rahimian

Teesside University, United Kingdom - ORCID: 0000-0001-7443-4723

Diptangshu Pandit

Teesside University, United Kingdom - ORCID: 0000-0001-7647-3443

Hannah Thomas

The Faraday Centre LTD, United Kingdom

Nigel Clark

The Faraday Centre LTD, United Kingdom

  1. Abulrub, A. H. G., Attridge, A. N., & Williams, M. A. (2011, 4-6 April 2011). Virtual reality in engineering education: The future of creative learning. 2011 IEEE Global Engineering Education Conference (EDUCON),
  2. Adami, P., Rodrigues, P. B., Woods, P. J., Becerik-Gerber, B., Soibelman, L., Copur-Gencturk, Y., & Lucas, G. (2021). Effectiveness of VR-based training on improving construction workers’ knowledge, skills, and safety behavior in robotic teleoperation. Advanced Engineering Informatics, 50, 101431. DOI: 10.1016/j.aei.2021.101431
  3. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., & Debevec, P. (2009, 12-13 Nov. 2009). Creating a Photoreal Digital Actor: The Digital Emily Project. 2009 Conference for Visual Media Production,
  4. Bot, J. A., Irschick, D. J., Grayburn, J., Lischer-Katz, Z., Golubiewski-Davis, K., & Ikeshoji-Orlati, V. (2019). Using 3D photogrammetry to create open-access models of live animals: 2D and 3D software solutions. Grayburn et al., eds. D, 3, 54-72.
  5. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2017). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4), 834-848.
  6. Cowie, N., & Alizadeh, M. (2022). The Affordances and Challenges of Virtual Reality for Language Teaching. International Journal of TESOL Studies, 4(3).
  7. Cui, B., Tao, W., & Zhao, H. (2021). High-Precision 3D Reconstruction for Small-to-Medium-Sized Objects Utilizing Line-Structured Light Scanning: A Review. Remote Sensing, 13(21), 4457. https://www.mdpi.com/2072-4292/13/21/4457
  8. Dewez, T. J. B., Girardeau-Montaut, D., Allanic, C., & Rohmer, J. (2016). FACETS : A CLOUDCOMPARE PLUGIN TO EXTRACT GEOLOGICAL PLANES FROM UNSTRUCTURED 3D POINT CLOUDS. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLI-B5, 799-804. DOI: 10.5194/isprs-archives-XLI-B5-799-2016
  9. Farella, E. M., Morelli, L., Rigon, S., Grilli, E., & Remondino, F. (2022). Analysing Key Steps of the Photogrammetric Pipeline for Museum Artefacts 3D Digitisation. Sustainability, 14(9), 5740. https://www.mdpi.com/2071-1050/14/9/5740
  10. Frost, A., Mirashrafi, S., Sánchez, C. M., Vacas-Madrid, D., Millan, E. R., & Wilson, L. (2023). Digital Documentation of Reflective Objects: A Cross-Polarised Photogrammetry Workflow for Complex Materials. In 3D Research Challenges in Cultural Heritage III: Complexity and Quality in Digitisation (pp. 131-155). Springer.
  11. Han, E., Nowak, K. L., & Bailenson, J. N. (2022). Prerequisites for Learning in Networked Immersive Virtual Reality.
  12. Kang, M. S., & An, Y.-K. (2021). Deep Learning-Based Automated Background Removal for Structural Exterior Image Stitching. Applied Sciences, 11(8), 3339. https://www.mdpi.com/2076-3417/11/8/3339
  13. Leberl, F., Irschara, A., Pock, T., Meixner, P., Gruber, M., Scholz, S., & Wiechert, A. (2010). Point Clouds. Photogrammetric Engineering & Remote Sensing, 76(10), 1123-1134. DOI: 10.14358/PERS.76.10.1123
  14. Loosemore, M., & Malouf, N. (2019). Safety training and positive safety attitude formation in the Australian construction industry. Safety Science, 113, 233-243. DOI: 10.1016/j.ssci.2018.11.029
  15. Moolman, J., Corkery, G., Walsh, J., & Morrissey-Tucker, S. (2022). THE USE OF COLLABORATIVE VIRTUAL ENVIRONMENTS (CVES) FOR ENGINEERING EDUCATION IN HIGHER EDUCATION INSTITUTIONS. EDULEARN22 Proceedings,
  16. Newton, S., Wang, R., & Lowe, R. (2015). Blended reality and presence. International Journal of Design Sciences & Technology, 21(2).
  17. Noya, N. C., García, Á. L., & Ramírez, F. C. (2015). Combining photogrammetry and photographic enhancement techniques for the recording of megalithic art in north-west Iberia. Digital Applications in Archaeology and Cultural Heritage, 2(2), 89-101. DOI: 10.1016/j.daach.2015.02.004
  18. Pan, Y., Braun, A., Brilakis, I., & Borrmann, A. (2022). Enriching geometric digital twins of buildings with small objects by fusing laser scanning and AI-based image recognition. Automation in Construction, 140, 104375. DOI: 10.1016/j.autcon.2022.104375
  19. Porter, S. T., Roussel, M., & Soressi, M. (2016). A Simple Photogrammetry Rig for the Reliable Creation of 3D Artifact Models in the Field: Lithic Examples from the Early Upper Paleolithic Sequence of Les Cottés (France). Advances in Archaeological Practice, 4(1), 71-86. DOI: 10.7183/2326-3768.4.1.71
  20. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18,
  21. Sacks, R., Perlman, A., & Barak, R. (2013). Construction safety training using immersive virtual reality. Construction Management and Economics, 31(9), 1005-1017. DOI: 10.1080/01446193.2013.828844
  22. Schaich, M., & Fritsch, D. (2013). Combined 3D scanning and photogrammetry surveys with 3D database support for archaeology and cultural heritage. A practice report on ArcTron’s information system aSPECT3D. Photogrammetric Week’13, 233-246.
  23. Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3549-3557.
  24. Stefan, H., Mortimer, M., Horan, B., & Kenny, G. (2023). Evaluating the preliminary effectiveness of industrial virtual reality safety training for ozone generator isolation procedure. Safety Science, 163, 106125. DOI: 10.1016/j.ssci.2023.106125
  25. Triantafyllou, V., Kotsopoulos, K. I., Tsolis, D., & Tsoukalos, D. (2022, 18-20 July 2022). Practical Techniques for Aerial Photogrammetry, Polygon Reduction and Aerial 360 Photography for Cultural Heritage Preservation in AR and VR Applications. 2022 13th International Conference on Information, Intelligence, Systems & Applications (IISA),
  26. Wang, J., Li, Z., Hu, W., Shao, Y., Wang, L., Wu, R., Ma, K., Zou, D., & Chen, Y. (2019). Virtual reality and integrated crime scene scanning for immersive and heterogeneous crime scene reconstruction. Forensic Science International, 303, 109943. DOI: 10.1016/j.forsciint.2019.109943
  27. Wang, R., Peethambaran, J., & Chen, D. (2018). LiDAR Point Clouds to 3-D Urban Models$:$ A Review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(2), 606-627. DOI: 10.1109/JSTARS.2017.2781132
  28. White, W. W., & Jung, M. J. (2022). Three-Dimensional Virtual Reality Spinal Cord Stimulator Training Improves Trainee Procedural Confidence and Performance. Neuromodulation: Technology at the Neural Interface. DOI: 10.1016/j.neurom.2022.03.005
PDF
  • Publication Year: 2023
  • Pages: 274-287

XML
  • Publication Year: 2023

Chapter Information

Chapter Title

A Framework for Realistic Virtual Representation for Immersive Training Environments.

Authors

Caolan Plumb, Farzad Pour Rahimian, Diptangshu Pandit, Hannah Thomas, Nigel Clark

DOI

10.36253/979-12-215-0289-3.26

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Bibliographic Information

Book Title

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

Book Subtitle

Managing the Digital Transformation of Construction Industry

Editors

Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Publisher Name

Firenze University Press

DOI

10.36253/979-12-215-0289-3

eISBN (pdf)

979-12-215-0289-3

eISBN (xml)

979-12-215-0257-2

Series Title

Proceedings e report

Series ISSN

2704-601X

Series E-ISSN

2704-5846

156

Fulltext
downloads

198

Views

Export Citation

1,361

Open Access Books

in the Catalogue

2,368

Book Chapters

3,870,371

Fulltext
downloads

4,536

Authors

from 943 Research Institutions

of 66 Nations

67

scientific boards

from 357 Research Institutions

of 43 Nations

1,249

Referees

from 381 Research Institutions

of 38 Nations