Contained in:
Book Chapter

Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios

  • Aqsa Sabir
  • Rahat Hussain
  • Syed Farhan Alam Zaidi
  • Akeem Pedro
  • Mehrtash Soltani
  • Dongmin Lee
  • Chansik Park

Computer vision-based safety monitoring requires machine learning models trained on generalized datasets covering various viewpoints, surface properties, and lighting conditions. However, capturing high-quality and extensive datasets for some construction scenarios is challenging at real job sites due to the risky nature of construction scenarios. Previous methods have proposed synthetic data generation techniques involving 2D background randomization with virtual objects in game-based engines. While there has been extensive work on utilizing 360-degree images for various purposes, no study has yet employed 360-degree images for generating synthetic data specifically tailored for construction sites. To improve the synthetic data generation process, this study proposes a 360-degree images-based synthetic data generation approach using Unity 3D game engine. The approach efficiently generates a sizable dataset with better dimensions and scaling, encompassing a range of camera positions with randomized lighting intensities. To check the effectiveness of our proposed method, we conducted a subjective evaluation, considering three key factors: object positioning, scaling in terms of object respective size, and the overall size of the generated dataset. The synthesized images illustrate the visual improvement in all three factors. By offering an improved data generation method for training safety-focused computer vision models, this research has the potential to significantly enhance the automation of the construction safety monitoring process, and hence, this method can bring substantial benefits to the construction industry by improving operational efficiency and reinforcing safety measures for workers

  • Keywords:
  • 360-Degree Images,
  • Computer Vision,
  • Synthetic Data Generation,
  • Game Engine,
  • Object Detection,
  • Construction Safety Monitoring,
+ Show More

Aqsa Sabir

Chung Ang University, Korea (the Republic of) - ORCID: 0009-0006-5459-909X

Rahat Hussain

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-6909-5189

Syed Farhan Alam Zaidi

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0003-2257-290X

Akeem Pedro

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-7884-5316

Mehrtash Soltani

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-5217-2010

Dongmin Lee

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0002-3176-5327

Chansik Park

Chung Ang University, Korea (the Republic of) - ORCID: 0000-0003-2256-300X

  1. Assadzadeh, A., Arashpour, M., Brilakis, I., Ngo, T., & Konstantinou, E. (2022). Vision-based excavator pose estimation using synthetically generated datasets with domain randomization. Automation in Construction, 134, 104089. DOI: 10.1016/j.autcon.2021.104089
  2. Barrera-Animas, A. Y., & Davila Delgado, J. M. (2023). Generating real-world-like labelled synthetic datasets for construction site applications. Automation in Construction, 151, 104850. DOI: 10.1016/j.autcon.2023.104850
  3. Borkman, S., Crespi, A., Dhakad, S., Ganguly, S., Hogins, J., Jhang, Y.-C., Kamalzadeh, M., Li, B., Leal, S., Parisi, P., Romero, C., Smith, W., Thaman, A., Warren, S., & Yadav, N. (2021). Unity Perception: Generate Synthetic Data for Computer Vision. DOI: 10.48550/arXiv.2107.04259
  4. Choi, W., & Pyun, S. (2021). Synthetic Training Data Generation for Fault Detection Based on Deep Learning. Geophysics and Geophysical Exploration, 24(3), 89–97. DOI: 10.7582/GGE.2021.24.3.089
  5. Frolov, V., Faizov, B., Shakhuro, V., Sanzharov, V., Konushin, A., Galaktionov, V., & Voloboy, A. (2022). Image Synthesis Pipeline for CNN-Based Sensing Systems. Sensors, 22(6), 2080. DOI: 10.3390/s22062080
  6. Kim, A., Lee, K., Lee, S., Song, J., Kwon, S., & Chung, S. (2022). Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding. Applied Sciences, 12(19), 10097. DOI: 10.3390/app121910097
  7. Kim, J., Kim, D., Shah, J., & Lee, S. (2022). Training a Visual Scene Understanding Model Only with Synthetic Construction Images. Computing in Civil Engineering 2021, 221–229. DOI: 10.1061/9780784483893.028
  8. Lee, J. G., Asce, S. M., Hwang, J., Chi, S., Asce, M., & Seo, J. (2022). Synthetic Image Dataset Development for Vision-Based Construction Equipment Detection. Computing in Civil Engineering, 36(5). DOI: 10.1061/(ASCE)CP.1943-5487.0001035
  9. Li, Y., Wei, H., Han, Z., Jiang, N., Wang, W., & Huang, J. (2022). Computer Vision-Based Hazard Identification of Construction Site Using Visual Relationship Detection and Ontology. Buildings, 12(6), 857.
  10. Neuhausen, M., Herbers, P., & König, M. (2020). Using synthetic data to improve and evaluate the tracking performance of construction workers on site. Applied Sciences (Switzerland), 10(14), 4948. DOI: 10.3390/app10144948
  11. Oh, X., Loh, L., Foong, S., Koh, Z. B. A., Ng, K. L., Tan, P. K., Toh, P. L. P., & Tan, U. X. (2021). Initialisation of Autonomous Aircraft Visual Inspection Systems via CNN-Based Camera Pose Estimation. Proceedings - IEEE International Conference on Robotics and Automation, 2021-May, 11047–11053. DOI: 10.1109/ICRA48506.2021.9561575
  12. Rho, J., Park, M., & Lee, H.-S. (2020). Automated construction progress management using computer vision-based CNN model and BIM. Korean Journal of Construction Engineering and Management, 21(5), 11–19.
  13. Sami Ur Rehman, M., Shafiq, M. T., & Ullah, F. (2022). Automated Computer Vision-Based Construction Progress Monitoring: A Systematic Review. Buildings, 12(7), 1037.
  14. Siu, C., Wang, M., & Cheng, J. C. P. (2022). A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection. Automation in Construction, 137, 104213. DOI: 10.1016/j.autcon.2022.104213
  15. Soltani, M. M., Zhu, Z., & Hammad, A. (2016). Automated annotation for visual recognition of construction resources using synthetic images. Automation in Construction, 62, 14–23. DOI: 10.1016/j.autcon.2015.10.002
  16. Sutjaritvorakul, T., Vierling, A., & Berns, K. (2020). Data-driven worker detection from load-view crane camera. Proceedings of the 37th International Symposium on Automation and Robotics in Construction, ISARC 2020: From Demonstration to Practical Use - To New Stage of Construction Robot, 864–871. DOI: 10.22260/isarc2020/0119
  17. Tohidifar, A., Fatemeh Saffari, S., & Kim, D. (2022). Fake it till you make it: Training Deep Neural Networks for Worker Detection using Synthetic Data. The 29th EG-ICE International Workshop on Intelligent Computing in Engineering, 386–396. DOI: 10.7146/AUL.455.C229
  18. Wei, Y., & Akinci, B. (2022). Synthetic Image Data Generation for Semantic Understanding in Everchanging Scenes Using BIM and Unreal Engine. Computing in Civil Engineering 2021, 934–941. DOI: 10.1061/9780784483893.115
  19. Wong, M. Z., Kunii, K., Baylis, M., Ong, W. H., Kroupa, P., & Koller, S. (2019). Synthetic dataset generation for object-to-model deep learning in industrial applications. PeerJ Computer Science, 2019(10), e222. DOI: 10.7717/peerj-cs.222
  20. Zhang, J., Fukuda, T., & Yabuki, N. (2022). Automatic generation of synthetic datasets from a city digital twin for use in the instance segmentation of building facades. Journal of Computational Design and Engineering, 9(5), 1737–1755. DOI: 10.1093/jcde/qwac086
PDF
  • Publication Year: 2023
  • Pages: 701-710

XML
  • Publication Year: 2023

Chapter Information

Chapter Title

Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios

Authors

Aqsa Sabir, Rahat Hussain, Syed Farhan Alam Zaidi, Akeem Pedro, Mehrtash Soltani, Dongmin Lee, Chansik Park

DOI

10.36253/979-12-215-0289-3.70

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Bibliographic Information

Book Title

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

Book Subtitle

Managing the Digital Transformation of Construction Industry

Editors

Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Publisher Name

Firenze University Press

DOI

10.36253/979-12-215-0289-3

eISBN (pdf)

979-12-215-0289-3

eISBN (xml)

979-12-215-0257-2

Series Title

Proceedings e report

Series ISSN

2704-601X

Series E-ISSN

2704-5846

83

Fulltext
downloads

122

Views

Export Citation

1,347

Open Access Books

in the Catalogue

2,262

Book Chapters

3,790,127

Fulltext
downloads

4,421

Authors

from 923 Research Institutions

of 65 Nations

65

scientific boards

from 348 Research Institutions

of 43 Nations

1,248

Referees

from 380 Research Institutions

of 38 Nations