Contained in:
Book Chapter

Predictive Safety Monitoring for Lifting Operations with Vision-Based Crane-Worker Conflict Prediction

  • Peter Kok-Yiu Wong
  • Chin Pok Lam
  • Yin Ni Lee
  • Chung Lam Ting
  • Jack C. P. Cheng
  • Pak Him Leung

Construction industry has reported among the highest accident and fatality rates over the past decade. In particular, crane lifting is a notably hazardous operation on construction sites, causing fatal accidents like workers being struck by the boom or objects fallen from tower cranes. Manual monitoring by on-site safety officers is labour-intensive and error-prone, while incorporating computer vision techniques into surveillance cameras would enable more automatic and continuous monitoring of construction site operations. However, existing studies for lifting safety mainly detect the presence of individual objects (e.g. workers, crane components), while a methodology is needed to predict their potential collision more proactively before accidents happen. This paper develops a vision-based framework for predictive lifting safety monitoring, including three modules: (1) object detection and classification: targeting at hook and lifting materials to enable danger zone estimation, along with workers and their personal protective equipment; (2) worker movement tracking and prediction: analyzing the historical moving trajectory of each unique worker to foresee his/her future movement in certain period ahead; (3) multi-level safety assessment: issuing predictive warning in real-time upon any crane-worker conflict foreseen. The proposed framework is applicable to real-time site video processing and enables end-to-end lifting safety monitoring with instant alerting upon unsafe scenarios observed. Importantly, the proposed framework predicts the future movement of workers to proactively identify potential site hazard, in order to trigger earlier safety alert for more timely decision-making. With a large video dataset capturing tower crane operations, the proposed framework demonstrates competitive accuracy and computational efficiency in crane-worker conflict prediction, validating its practicality for real-time lifting safety monitoring

  • Keywords:
  • Computer Vision; Construction Safety Monitoring; Crane-Worker Conflict Prediction; Deep Learning; Predictive Safety Assessment; Trajectory Tracking,
+ Show More

Peter Kok-Yiu Wong

The Hong Kong University of Science and Technology, Hong Kong - ORCID: 0000-0003-1758-675X

Chin Pok Lam

University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong

Yin Ni Lee

University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong

Chung Lam Ting

University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong

Jack C. P. Cheng

University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong - ORCID: 0000-0002-1722-2617

Pak Him Leung

AutoSafe Limited, Hong Kong - ORCID: 0000-0001-8627-6216

  1. Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V. (2018). Computer vision and deep learning techniques for pedestrian detection and tracking: A survey. Neurocomputing, 300, 17-33. DOI: 10.1016/j.neucom.2018.01.092
  2. Cheng, J. P., Wong, P. K. Y., Luo, H., Wang, M., & Leung, P. H. (2022). Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification. Automation in Construction, 139, 104312. DOI: 10.1016/j.autcon.2022.104312
  3. Chian, E. Y. T., Goh, Y. M., Tian, J., & Guo, B. H. (2022). Dynamic identification of crane load fall zone: A computer vision approach. Safety science, 156, 105904. DOI: 10.1016/j.ssci.2022.105904
  4. Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., & An, W. (2018). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in construction, 85, 1-9. DOI: 10.1016/j.autcon.2017.09.018
  5. Golovina, O., Teizer, J., & Pradhananga, N. (2016). Heat map generation for predictive safety planning: Preventing struck-by and near miss interactions between workers-on-foot and construction equipment. Automation in construction, 71, 99-115. DOI: 10.1016/j.autcon.2016.03.008
  6. HKSARG Development Bureau (2020). Lifting Safety Handbook. Construction Safety Week 2020. [Online]. Available: https://www.cic.hk/files/page/51/Lifting%20Safety%20Handbook%20%E5%90%8A%E9%81%8B%E5%AE%89%E5%85%A8%E6%89%8B%E5%86%8A.pdf. [Accessed: Oct. 29, 2022]
  7. HKSARG Labour Department (2018). Occupational Safety and Health Statistics: Bulletin Issue No. 18. [Online]. Available: https://www.labour.gov.hk/eng/osh/pdf/Bulletin2017_issue18_eng.pdf. [Accessed: 18-Dec-2022]
  8. Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer I., Wojna Z., Song Y., Guadarrama S., & Murphy, K. (2017). Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7310-7311). DOI: 10.1109/cvpr.2017.351
  9. Jeelani, I., Asadi, K., Ramshankar, H., Han, K., & Albert, A. (2021). Real-time vision-based worker localization & hazard detection for construction. Automation in Construction, 121, 103448. DOI: 10.1016/j.autcon.2020.103448
  10. Kim D., Liu M., Lee S. H., & Kamat V. R. (2019). Remote proximity monitoring between mobile construction resources using camera-mounted uavs. Automation in Construction, vol. 99, pp. 168–182. DOI: 10.1016/j.autcon.2018.12.014
  11. Liu, W., Meng, Q., Li, Z., & Hu, X. (2021). Applications of computer vision in monitoring the unsafe behavior of construction workers: Current status and challenges. Buildings, 11(9), 409. DOI: 10.3390/buildings11090409
  12. Luo, H., Wang, M., Wong, P. K. Y., Tang, J., & Cheng, J. C. (2021). Construction machine pose prediction considering historical motions and activity attributes using gated recurrent unit (GRU). Automation in Construction, 121, 103444. DOI: 10.1016/j.autcon.2020.103444
  13. Luo, X., Li, H., Wang, H., Wu, Z., Dai, F., & Cao, D. (2019). Vision-based detection and visualization of dynamic workspaces. Automation in Construction, 104, 1-13. DOI: 10.1016/j.autcon.2019.04.001
  14. Memarzadeh, M., Golparvar-Fard, M., & Niebles, J. C. (2013). Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Automation in Construction, 32, 24-37. DOI: 10.1016/j.autcon.2012.12.002
  15. Nath, N. D., Behzadan, A. H., & Paal, S. G. (2020). Deep learning for site safety: Real-time detection of personal protective equipment. Automation in Construction, 112, 103085. DOI: 10.1016/j.autcon.2020.103085
  16. Shafique, M., & Rafiq, M. (2019). An overview of construction occupational accidents in Hong Kong: A recent trend and future perspectives. Applied Sciences, 9(10), 2069. DOI: 10.3390/app9102069
  17. Son, H., Seong, H., Choi, H., & Kim, C. (2019). Real-time vision-based warning system for prevention of collisions between workers and heavy equipment. Journal of Computing in Civil Engineering, 33(5), 04019029. DOI: 10.1061/(ASCE)CP.1943-5487.0000845
  18. U.S. Bureau of Labor Statistics (BLS) (2018). National Census of Fatal Occupational Injuries in 2017. [Online]. Available: www.bls.gov/iif/oshcfoil.htm. [Accessed: 18-Dec-2022]
  19. Wojke, N., Bewley, A., & Paulus, D. (2017, September). Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP) (pp. 3645-3649). IEEE. DOI: 10.1109/icip.2017.8296962
  20. Wong, P. K. Y., Luo, H., Wang, M., Leung, P. H., & Cheng, J. C. (2021). Recognition of pedestrian trajectories and attributes with computer vision and deep learning techniques. Advanced Engineering Informatics, 49, 101356. DOI: 10.1016/j.aei.2021.101356
  21. Zhang, M., Cao, Z., Yang, Z., & Zhao, X. (2020). Utilizing computer vision and fuzzy inference to evaluate level of collision safety for workers and equipment in a dynamic environment. Journal of Construction Engineering and Management, 146(6), 04020051. DOI: 10.1061/(ASCE)CO.1943-7862.0001802
PDF
  • Publication Year: 2023
  • Pages: 648-656

XML
  • Publication Year: 2023

Chapter Information

Chapter Title

Predictive Safety Monitoring for Lifting Operations with Vision-Based Crane-Worker Conflict Prediction

Authors

Peter Kok-Yiu Wong, Chin Pok Lam, Yin Ni Lee, Chung Lam Ting, Jack C. P. Cheng, Pak Him Leung

DOI

10.36253/979-12-215-0289-3.64

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Bibliographic Information

Book Title

CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality

Book Subtitle

Managing the Digital Transformation of Construction Industry

Editors

Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi

Peer Reviewed

Publication Year

2023

Copyright Information

© 2023 Author(s)

Content License

CC BY-NC 4.0

Metadata License

CC0 1.0

Publisher Name

Firenze University Press

DOI

10.36253/979-12-215-0289-3

eISBN (pdf)

979-12-215-0289-3

eISBN (xml)

979-12-215-0257-2

Series Title

Proceedings e report

Series ISSN

2704-601X

Series E-ISSN

2704-5846

122

Fulltext
downloads

251

Views

Export Citation

1,340

Open Access Books

in the Catalogue

2,191

Book Chapters

3,790,127

Fulltext
downloads

4,399

Authors

from 923 Research Institutions

of 65 Nations

64

scientific boards

from 348 Research Institutions

of 43 Nations

1,247

Referees

from 380 Research Institutions

of 38 Nations