- CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality
- Edited by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi
Predictive Safety Monitoring for Lifting Operations with Vision-Based Crane-Worker Conflict Prediction
- Peter Kok-Yiu Wong
- Chin Pok Lam
- Yin Ni Lee
- Chung Lam Ting
- Jack C. P. Cheng
- Pak Him Leung
- © 2023 Author(s) |
- CC BY-NC 4.0
- DOI: 10.36253/979-12-215-0289-3.64
Construction industry has reported among the highest accident and fatality rates over the past decade. In particular, crane lifting is a notably hazardous operation on construction sites, causing fatal accidents like workers being struck by the boom or objects fallen from tower cranes. Manual monitoring by on-site safety officers is labour-intensive and error-prone, while incorporating computer vision techniques into surveillance cameras would enable more automatic and continuous monitoring of construction site operations. However, existing studies for lifting safety mainly detect the presence of individual objects (e.g. workers, crane components), while a methodology is needed to predict their potential collision more proactively before accidents happen. This paper develops a vision-based framework for predictive lifting safety monitoring, including three modules: (1) object detection and classification: targeting at hook and lifting materials to enable danger zone estimation, along with workers and their personal protective equipment; (2) worker movement tracking and prediction: analyzing the historical moving trajectory of each unique worker to foresee his/her future movement in certain period ahead; (3) multi-level safety assessment: issuing predictive warning in real-time upon any crane-worker conflict foreseen. The proposed framework is applicable to real-time site video processing and enables end-to-end lifting safety monitoring with instant alerting upon unsafe scenarios observed. Importantly, the proposed framework predicts the future movement of workers to proactively identify potential site hazard, in order to trigger earlier safety alert for more timely decision-making. With a large video dataset capturing tower crane operations, the proposed framework demonstrates competitive accuracy and computational efficiency in crane-worker conflict prediction, validating its practicality for real-time lifting safety monitoring
- Keywords:
- Computer Vision; Construction Safety Monitoring; Crane-Worker Conflict Prediction; Deep Learning; Predictive Safety Assessment; Trajectory Tracking,
The Hong Kong University of Science and Technology, Hong Kong - ORCID: 0000-0003-1758-675X
University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong
University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong
University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong
University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong - ORCID: 0000-0002-1722-2617
AutoSafe Limited, Hong Kong - ORCID: 0000-0001-8627-6216
- Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V. (2018). Computer vision and deep learning techniques for pedestrian detection and tracking: A survey. Neurocomputing, 300, 17-33. DOI: 10.1016/j.neucom.2018.01.092
- Cheng, J. P., Wong, P. K. Y., Luo, H., Wang, M., & Leung, P. H. (2022). Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification. Automation in Construction, 139, 104312. DOI: 10.1016/j.autcon.2022.104312
- Chian, E. Y. T., Goh, Y. M., Tian, J., & Guo, B. H. (2022). Dynamic identification of crane load fall zone: A computer vision approach. Safety science, 156, 105904. DOI: 10.1016/j.ssci.2022.105904
- Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., & An, W. (2018). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in construction, 85, 1-9. DOI: 10.1016/j.autcon.2017.09.018
- Golovina, O., Teizer, J., & Pradhananga, N. (2016). Heat map generation for predictive safety planning: Preventing struck-by and near miss interactions between workers-on-foot and construction equipment. Automation in construction, 71, 99-115. DOI: 10.1016/j.autcon.2016.03.008
- HKSARG Development Bureau (2020). Lifting Safety Handbook. Construction Safety Week 2020. [Online]. Available: https://www.cic.hk/files/page/51/Lifting%20Safety%20Handbook%20%E5%90%8A%E9%81%8B%E5%AE%89%E5%85%A8%E6%89%8B%E5%86%8A.pdf. [Accessed: Oct. 29, 2022]
- HKSARG Labour Department (2018). Occupational Safety and Health Statistics: Bulletin Issue No. 18. [Online]. Available: https://www.labour.gov.hk/eng/osh/pdf/Bulletin2017_issue18_eng.pdf. [Accessed: 18-Dec-2022]
- Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer I., Wojna Z., Song Y., Guadarrama S., & Murphy, K. (2017). Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7310-7311). DOI: 10.1109/cvpr.2017.351
- Jeelani, I., Asadi, K., Ramshankar, H., Han, K., & Albert, A. (2021). Real-time vision-based worker localization & hazard detection for construction. Automation in Construction, 121, 103448. DOI: 10.1016/j.autcon.2020.103448
- Kim D., Liu M., Lee S. H., & Kamat V. R. (2019). Remote proximity monitoring between mobile construction resources using camera-mounted uavs. Automation in Construction, vol. 99, pp. 168–182. DOI: 10.1016/j.autcon.2018.12.014
- Liu, W., Meng, Q., Li, Z., & Hu, X. (2021). Applications of computer vision in monitoring the unsafe behavior of construction workers: Current status and challenges. Buildings, 11(9), 409. DOI: 10.3390/buildings11090409
- Luo, H., Wang, M., Wong, P. K. Y., Tang, J., & Cheng, J. C. (2021). Construction machine pose prediction considering historical motions and activity attributes using gated recurrent unit (GRU). Automation in Construction, 121, 103444. DOI: 10.1016/j.autcon.2020.103444
- Luo, X., Li, H., Wang, H., Wu, Z., Dai, F., & Cao, D. (2019). Vision-based detection and visualization of dynamic workspaces. Automation in Construction, 104, 1-13. DOI: 10.1016/j.autcon.2019.04.001
- Memarzadeh, M., Golparvar-Fard, M., & Niebles, J. C. (2013). Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Automation in Construction, 32, 24-37. DOI: 10.1016/j.autcon.2012.12.002
- Nath, N. D., Behzadan, A. H., & Paal, S. G. (2020). Deep learning for site safety: Real-time detection of personal protective equipment. Automation in Construction, 112, 103085. DOI: 10.1016/j.autcon.2020.103085
- Shafique, M., & Rafiq, M. (2019). An overview of construction occupational accidents in Hong Kong: A recent trend and future perspectives. Applied Sciences, 9(10), 2069. DOI: 10.3390/app9102069
- Son, H., Seong, H., Choi, H., & Kim, C. (2019). Real-time vision-based warning system for prevention of collisions between workers and heavy equipment. Journal of Computing in Civil Engineering, 33(5), 04019029. DOI: 10.1061/(ASCE)CP.1943-5487.0000845
- U.S. Bureau of Labor Statistics (BLS) (2018). National Census of Fatal Occupational Injuries in 2017. [Online]. Available: www.bls.gov/iif/oshcfoil.htm. [Accessed: 18-Dec-2022]
- Wojke, N., Bewley, A., & Paulus, D. (2017, September). Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP) (pp. 3645-3649). IEEE. DOI: 10.1109/icip.2017.8296962
- Wong, P. K. Y., Luo, H., Wang, M., Leung, P. H., & Cheng, J. C. (2021). Recognition of pedestrian trajectories and attributes with computer vision and deep learning techniques. Advanced Engineering Informatics, 49, 101356. DOI: 10.1016/j.aei.2021.101356
- Zhang, M., Cao, Z., Yang, Z., & Zhao, X. (2020). Utilizing computer vision and fuzzy inference to evaluate level of collision safety for workers and equipment in a dynamic environment. Journal of Construction Engineering and Management, 146(6), 04020051. DOI: 10.1061/(ASCE)CO.1943-7862.0001802
Chapter Information
Chapter Title
Predictive Safety Monitoring for Lifting Operations with Vision-Based Crane-Worker Conflict Prediction
Authors
Peter Kok-Yiu Wong, Chin Pok Lam, Yin Ni Lee, Chung Lam Ting, Jack C. P. Cheng, Pak Him Leung
DOI
10.36253/979-12-215-0289-3.64
Peer Reviewed
Publication Year
2023
Copyright Information
© 2023 Author(s)
Content License
Metadata License
Bibliographic Information
Book Title
CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality
Book Subtitle
Managing the Digital Transformation of Construction Industry
Editors
Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi
Peer Reviewed
Publication Year
2023
Copyright Information
© 2023 Author(s)
Content License
Metadata License
Publisher Name
Firenze University Press
DOI
10.36253/979-12-215-0289-3
eISBN (pdf)
979-12-215-0289-3
eISBN (xml)
979-12-215-0257-2
Series Title
Proceedings e report
Series ISSN
2704-601X
Series E-ISSN
2704-5846