In order to identify the constraints that affect the application of BIM technology in prefabricated buildings, 11 influencing factors were selected from the four dimensions of policy, economy, market, and technology and organized into a factor set. Firstly, the decision-making trial and evaluation laboratory (DEMATEL) method was used to evaluate the centrality and causality among the constraints; then, the interpretive structural modeling (ISM) was employed to establish a multi-level structure model of the constraints; finally, the cross-impact matrix multiplication applied to classification (MICMAC) method was utilized to obtain the driver-dependency relationship matrix of the constraints. The three modes were coupled to establish a composite mode, and on this basis, the logical relationship, hierarchical relationship, and relative importance among the constraints were clarified. The results show that the resistance of the design and construction parties is the primary reason for the restraint of BIM technology in prefabricated buildings, and the lack of innovation and professional talents are the fundamental reasons.