Rapid 3D Energy Performance Modeling of Existing Buildings using ...
Construction Research Congress 2012 © ASCE 2012
Rapid 3D Energy Performance Modeling of Existing Buildings using Thermal and Digital Imagery Youngjib Ham1 and Mani Golparvar-Fard2 1
Vecellio Pre-doctoral Fellow, Vecellio Construction Eng. and Mgmt., Via Dept. of Civil and Env. Eng., and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH (540) 235-6532; FAX (540) 231-7532; email: [email protected]
2
Assistant Professor, Vecellio Construction Eng. and Mgmt., Via Dept. of Civil and Env. Eng., and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH (540) 231-7255; FAX (540) 231-7532; email: [email protected]
ABSTRACT Rapid energy modeling is a streamlined process to capture, model, and analyze building energy performance. Timely assessment of existing building energy performance helps owners and facility managers to identify potential areas for better retrofit, and meet environmental and economic goals. Despite the increasing attention to building energy performance, the current process of energy data collection and modeling is time-consuming and requires certain level of expertise. In order to facilitate the process, this paper presents a new approach for actual energy performance modeling of existing buildings using digital and thermal imagery. First, using an image-based 3D reconstruction pipeline which consists of Structure-fromMotion, Multi-View Stereo, and Voxel Coloring/Labeling, the current geometrical condition of a building is captured. Subsequently, using a new 3D thermal modeling algorithm, a dense thermal point cloud model of the existing building is reconstructed in 3D. Finally, the reconstructed 3D building and thermal models are automatically superimposed within a single environment. Within the resulting 3D spatio-thermal point cloud model, temperature values can be queried and visualized at point level. The proposed method is validated on several rooms in an existing instructional facility. The underlying modeling process, and the potential benefits from converting digital and thermal imagery into ubiquitous sensors and reporters of building energy use are discussed in detail. INTRODUCTION The building sector uses about 40 percent of the total energy produced in the United States (DOE 2010). When looking at the life-cycle energy consumption of the U.S. buildings, nearly three quarters of produced electricity is consumed in operating buildings (DOE 2010). According to a recent report by the Energy Information Administration (EIA 2010), between 2010 and 2030, the increase in building sector’s energy consumption (7.16 Quadrillion Btu) will be more than those of the industrial and transportation sectors (4.05 and 3.36 Quadrillion Btu respectively). This energy is mainly produced from burning fossil fuels, making the building sector to be the largest producer of Greenhouse Gas emissions and the prime leading contributor to anthropogenic climate change (EIA 2010; Architecture2030 2010). Considering the significant energy consumption and the environmental impacts, government agencies are proposing incentive-based regulations to increase energy efficiency in residential and commercial buildings (e.g., 2011 Better Buildings Initiative). Currently several energy efficiency programs such as LEED and ENERGY STAR are becoming
991
Construction Research Congress 2012 © ASCE 2012
prevalent in the U.S. (DOE 2010). As a result of these mandatory and voluntary programs, a larger number of new buildings are being rigorously constructed to meet the requirement of these energy codes and efficiency programs. Nonetheless, a large number of existing buildings do not meet these standards, and their average age is rising. Retrofitting these buildings is important since they continuously undergo degradation over their service life. According to the U.S. Department of Energy (DOE 2010), 87 percent of overall residential buildings and 74 percent of total commercial buildings were built before 2000, and about 150 billion square feet of the existing buildings (i.e., roughly half of the entire building stock in the U.S.) will require renovations to meet the rigorous energy standards over the next 30 years (Gould and Hosey 2009). Faulty behaviors in existing buildings can alone account for 2~11 percent of the total building energy use (Roth et al 2005). Retrofitting existing buildings can potentially cut such excessive operational costs. Since any retrofit alternative directly translates into cost, building occupants are increasingly interested in identifying potential retrofit candidates of their buildings. Energy performance modeling of existing buildings is essentially an empirical exercise that relies on good observations to capture as-is building conditions. Once buildings are operating, the actual energy performance typically deviates from the baseline model, which is mainly due to the assumptions used during the modeling process. If the baseline energy model can closely represent the actual building energy performance, then it is more likely to provide a reliable source for the analysis of retrofit alternatives. To get a good match between the baseline models and the measured energy performance, modelers need to calibrate the baseline model parameters to exactly represent the actual building energy performance within an acceptable threshold. This is currently a widely accepted modeling process for analyzing existing buildings (Heo et al 2012; Ahmad and Culp 2006). Despite the effectiveness, several challenges can adversely impact the benefits: (1) Current energy modeling practices using existing building energy performance simulation tools (e.g., EnergyPlus, Ecotect, and eQuest) are timeconsuming and labor-intensive. First, building conditions are collected from the asplanned CAD models or by using traditional field surveying techniques (e.g., tape measurement or application of Total Stations). Subsequently, modelers should manually develop 3D energy models of existing buildings with the geometric information and various loads (e.g., weather data and occupancy schedules). Finally, extra time and effort for manual calibration of the model is needed to minimize the differences between the baseline models and the measured energy performance to a reasonably small level. Hence, the process of constructing models often requires weeks to months, and is therefore, often restricted to only high-budget projects (Autodesk 2011); (2) Existing energy models typically do not reflect a realistic picture of buildings as they are used. For example during modeling, all areas in a building are typically assumed to have similar insulation conditions, and each building element is assumed to have similar material conditions (e.g., degradation level). Hence, these tools primarily focus on comprehensive building operation (e.g., HVAC systems) or appliance-level energy use in a given thermal zone, and as a result cannot represent specific building areas in need for retrofit (e.g., construction defects or building
992
Construction Research Congress 2012 © ASCE 2012
degradation over time). Consequently, these approaches may not represent the accurate actual energy performance. (3) Current practices require certain level of skill and experience in energy modeling. According to a recent survey from energy modeling stakeholders (Tupper et al. 2011), one of the major bottlenecks of the current practice was identified as the difficulty in translating building geometrical and energy information into proper inputs for energy modeling. Overall, current labor-intensive and often inaccurate methods can delay the decision-making process by not delivering the necessary information to owner and stakeholders in a timely manner. Recent data collection technologies for building energy modeling (such as laser scanners and environmental sensor networks) are promising to eliminate nonvalue adding tasks within manual building and energy data collection process, and reduce time in modeling. However, they have severe practical challenges as they are not necessarily adaptable to all existing building conditions. For example, application of laser scanners for building data collection can be very costly, requires manual postprocessing of the data to remove noise and at least two experts to operate the device. The results can also suffer from mixed pixel phenomena within dynamic building environments (Golparvar-Fard et al. 2011, Kiziltas et al. 2008). Installation of environmental sensors for energy data collection can also be expensive and time consuming to set up for many existing buildings. To address these inefficiencies, this paper presents a new method for rapid energy performance modeling of existing buildings. The proposed method leverages recent image-based 3D modeling approaches as well as thermal cameras. By using inexpensive and often existing digital images in addition to thermal imagery, our proposed method automatically generates 3D spatio-thermal models, and enables owners and facility managers to quickly get the actual energy performance data for their existing buildings. In the following sections, first the state of knowledge in the areas of 3D thermal modeling is briefly overviewed. Next, the research objectives and methodologies are presented. Finally, we present preliminary results as a proof of concept for the proposed method, and conclude with a discussion on the potential benefits and limitations. BACKGROUND AND RELATED WORK Thermography - Thermography is defined as detecting and measuring variations in heat emitted by an object and transforming them into visible images (Eads et al. 2000). In terms of building energy performance modeling, thermography is a robust tool in recording, analyzing, and reporting actual energy performance of existing buildings. Thermal images from buildings are directly influenced by the building energy performance such as energy transfer through building elements (e.g., thermal bridges) or space heating and cooling energy related to HVAC systems. Currently the majority of applications of thermal images (e.g., thermographic inspections) are based on 2D images of the scene. Since these images can only show what is in their field of view and are not geo-tagged, they are not sufficient to provide a whole picture of energy performance in a given space. To advance both the measurement and the interpretation of energy performance, there is a need to consider 3D thermal modeling. A 3D model containing thermal information enables to recognize how
993
Construction Research Congress 2012 © ASCE 2012
temperature values are spatially distributed in a given space in a holistic manner. 3D thermal modeling – A number of previous research studies in both computer science as well as building construction domains have proposed several new methods for 3D thermal modeling. Stockton (2010) suggested a 3D thermography monitoring system to manage power consumption, cooling, and IT operations in a data center. The results show the effectiveness of 3D thermography, yet due to the manual process involved in creating image mosaics and performing texture-mappings, the proposed approach is time-consuming and labor-intensive. Cho and Wang (2011) introduced a 3D thermal modeling using a LIDAR scanner and a thermal camera. The 3D geometrical information of existing building envelopes are first collected using the laser scanner, and then temperatures values are mapped to building exterior surfaces using the thermal camera. Similarly, Nüchter (2012) proposed constructing 3D thermal models of indoor environments using a robot equipped with a laser scanner and a thermal camera. To collect the building and energy data, these two studies require a large and dedicated equipment to support both a laser scanner and a thermal camera at the same time. However, due to complex and confined indoor environment, this approach may not be easily applicable to interior spaces. Moreover, laser scanning does not provide any semantic information of the appearance of the building elements which is attributed to the lack of features in Cartesian point clouds generated by laser scanners. Although several previous studies from computer science (Pelagottia et al. 2009; Gonzalez-Aguilera et al. 2010) generally yield accurate 3D thermal models, yet they only focused on confined isolated objects or single image pairs using multiple thermal cameras to build 3D thermal models (e.g., a combination of one digital camera and two thermal cameras or stereo thermal camera systems). These approaches would require building auditors to acquire more thermal cameras which comes at a higher price and may not significantly add value. OVERVIEW OF THE PROPOSED 3D ENERGY MODELING Our work explores the application of 2D digital and thermal imagery collected with a single thermal camera. The proposed method models current building geometrical information and visualizes actual energy performance at the level of both building elements and various spaces within a single virtual 3D environment. Given a collection of unordered and uncalibrated digital and thermal imagery, our goal is to automatically reconstruct a 3D model and assign temperature values to each point within the 3D reconstructed scene. In our approach, the digital images are capturing the as-is condition of existing buildings, and the thermal imagery are capturing the actual energy performance. Since our proposed method automatically generates 3D actual energy performance models with a large number of unordered digital and thermal images, it has the benefit of streamlining the modeling process. In addition, due to the portability of one single thermal camera (integrated with a digital camera) and its mobility in an indoor environment, it is suitable for building interior surveys. Furthermore, our work makes it possible to quickly and cost-effectively understand the actual thermal performance by help of 3D reconstructed semantically-rich model. Thus, the word ‘rapid’ in this study not only refers to the modeling process itself (i.e., which is streamlined and does not require expertise and manual process), but also the quick screening method to identify potential areas required for more detailed energy
994
Construction Research Congress 2012 © ASCE 2012
annalysis. In the followiing, the research metthodology aand validattion are furrther deetailed. Severaal studies suuch as (Auttodesk 20111) have alreeady provenn the benefitts of im mage-based 3D reconstrruction com mpared to coonventionall surveys orr laser scannning appproaches (e.g., mannual or suupervised approachess similar to Autodeesk’s Im mageModeleer (2011) oor PhotoMoodeler (2011)). Figure 1 shows an example of theermal and digital d imagges. Both off these sourcces of imagges were capptured usingg the sam me camera in Figure 1a. Imagess labeled with ‘b’ in F Figure 1 show the therrmal chharacteristicss of the sam me areas captured in ‘c’..
F Figure 1. Th hermal and d digital imaages in exissting buildin ngs Figurre 2 presentts an overviiew of data and processs in our prooposed metthod. Thhe followingg describes eeach step w within our appproach:
F Figure 2. O Overview off 3D energyy performan nce modelin ng Im mage-based d 3D recon nstruction – Our method initiaally takes unordered and unncalibrated digital im mages and implemennts a streaamlined im mage-based 3D recconstructionn algorithm which conssists of Struucture-from--Motion (SffM) (GolparrvarFaard et al. 2009, Snaveely et al. 20008), Multii-View Sterreo (MVS) (Furukawa and Poonce 2009),, and Voxell Coloring/L Labeling (V VCL) (Golpparvar-Fard et al. 20111) to creeate dense 3D buildingg point cloud models of the scenne. To geneerate 3D spatiotheermal modeels, thermal and digitall images need to be coo-registered to map therrmal vaalues to the reconstructe r ed 3D scenee. To that ennd, our initiaal approach was to form m the Eppipolar geom metry (Harttley and Zissserman 20004) betweenn every paiir of digital and theermal images and com mpute the Fuundamental matrix betw ween each ppair. In ordeer to auutomate thiss process forr all pairs, w we conductted various exhaustive experimentts on
995
Construction Research Congress 2012 © ASCE 2012
996
thee state-of-tthe-art invvariant featture annd corner detectionn algorithhms inccluding SIF FT, ASIFT, SURF, MS SR, annd Harris Corner detection to invvestigate w whether featuure points can c Figu ure 3. Therm mal cameraa calibration n bee detected and a matchedd across thhese paairs. In almoost all casess, these algoorithms worked poorlyy or did nott work at alll (in moost cases reesulted in deetection of lless than 100 correspondding points)). This is duue to typpical low rresolution of o thermal iimages, lackk of distincct features, and signifiicant raddial distortiion in therm mal imageryy. Thermal cameras usse gradient color codinng to caapture therm mal performance whichh smoothen all surface intensities, and as a reesult theere is no disstinct featurre that can bbe detectedd. To solve tthis challengge, we com mpute thee intrinsic pparameters oof the therm mal camera as a follows: Th hermal cam mera interrnal calibrration – Inn order to calibrate ddigital cameeras, caalibration rigs, in form m of checkkerboard, arre typicallyy used. How wever, the low ressolution thhermal cam meras whichh can onlyy detect thhermal diffferences caannot disstinctly disttinguish inteersecting coorners of a ccalibrating checkerboar c rd. Hence inn our woork, we creeate a therm mal calibrattion rig (5550mm x 7000mm) usingg 42 small L LED ligghts locatedd on the inteersections of the conveentional checkerboard ((See Figure 3a). Prrior to calibbration, the temperaturee range for thermal maap color coding is fixeed to maake sure each gradientt of color is correspondding to an aabsolute tem mperature vaalue. Onnce the lighhts are on, thhey will geenerate heatss, and the thhermal cam mera which have h theermal sensitivity of 0.005°C can eaasily distincct these corrners (See F Figure 3b ~ 3e). Thhrough this process, thee thermal caamera is intternally calibbrated (i.e. the focal length ( ) and raadial distorttion parameeters ( and oof the therm mal cameraa are caalculated). N Next, assumiing a pinhole camera m model, the loocal coordinnates of therrmal im magery denooted as aare undistortted by usingg: , , (1) In thiis equation, raddial distortioon:
is a fuunction thatt computes a scaling facctor to undoo the 1.0
‖ ‖
‖ ‖
(2)
This is an imporrtant step in our process, as almostt all thermal cameras cause siggnificant raddial distortion due to thheir wide-anngle lens whhich is usedd to increasee the fieeld of view and minim mize the tim me required for data coollection. Foor most therrmal caameras, this process onlly needs to bbe done oncce and does not need too be repeatedd for diffferent data collections since the thhermal cameera lens is fiixed. Deense 3D th hermal modeling – Next, N we exxtract the eextrinsic parrameters off the theermal cameeras (i.e., cam mera locatioon ( ) and orientation o ( )) from thhe results off the SffM step connducted on ccorrespondiing digital images taken in the sam me locationn and orientation with w the therrmal camerra (this appproach is vaalid for moost new therrmal mages caameras as tthey have the abilityy to capturre both diggital and tthermal im sim multaneouslly). Based oon the intrinnsic and exttrinsic cameera parametters, the therrmal caamera projecction matrixx ( for ppreviously undistorted thermal im mages is forrmed ussing:
Construction Research Congress 2012 © ASCE 2012
0 0 0
0
997
/2 /2 1
(3)
wherein and are the width and the height of the thermal images. The undistorted thermal images along with their camera projection matrices are fed into the dense 3D reconstruction steps (MVS and VCL) and yield a dense 3D thermal model. Integrated visualization of digital and thermal imagery and models – In our work, we have modified the D4AR – 4D Augmented Reality Model viewer (Golparvar-Fard et al. 2009) to enable joint visualization of digital and thermal 3D point cloud models and imagery. The resulting 3D spatio-thermal models are represented using the following data structures: , ,.., which consist of a 3D location and a A set of digital points Ƥ color combined from one of the digital images where the point was observed. , ,.., A set of digital cameras Ƈ which consist of a 3D location and a A set of thermal points Ƥ , ,.., color combined from one of the thermal images where the point was observed. A set of thermal cameras Ƈ , ,.., A mapping between the points and the cameras that observe them. EXPERIMENT RESULTS Table 1. Camera Technical Data AND DISCUSSIONS Camera Technical Items Characteristics Hardware – The digital and Built-in Digital Camera 2048 × 1536 pixels thermal images were captured Thermal Image Resolution 320 × 240 pixels using an E60 thermal camera Thermal Sensitivity 0.05°C from FLIR Systems AB, Thermal Measurement Accuracy ±3.6°F(±2°C) which has a built-in digital Digital Camera Field of View (FOV) 53° × 41° 25° × 19° camera. Table 1 reports the Thermal Camera FOV Minimum Focus Distance 1.31 ft.(0.4m) camera specifications. Data Collection – To test the proposed method, we performed experiments in an office room of an existing instructional facility on campus of Virginia Tech. The visual data were collected during a morning and under natural daylight settings. Results – Table 2 shows the results of our experiment. Figure 4a, b, c, and f represent the dense 3D reconstructed scenes from digital and thermal images, and illustrate one of the locations and orientations of the camera registered in a virtual 3D environment. Once a registered camera is visited (Figure 4d and g), the frustum of the camera is automatically texture-mapped with a full resolution of the image used to capture it (Figure 4e and h). Thus, the user can visually and interactively acquire information related to thermal performance Table 2. Experiment Outcomes and building condition of the Results Values 429 given area from both digital The # of Digital Images 429 and thermal imagery. Figure The # of Thermal Images 2,064,662 points 5a shows the result of Density of Thermal 3D Point Cloud Density of Building 3D Point Cloud 8,488,888 points superimposition of a 3D Computation Time ~ 6 hours building model over a 3D Benchmarked on an Intel(R) Core(TM) i7 960 with 24GBs of RAM
Construction Research Congress 2012 © ASCE 2012
theermal modeel based on the same coordinate c ssystem. A 33D spatio-thhermal moddel is auutomaticallyy texture-maapped with a full resoluution of therrmal image (Figure 5b)). 3D sem mantically-rrich reconsttructed moddels from diggital imagess allow to beetter undersstand hoow temperatture values are spatiallly distributted with seemantic infoormation off the givven space ffor a givenn point of tiime. More experimenttal results ccan be founnd at ww ww.raamac..cee.vt.edu/eepar.
Figure 4. Dense 3D D building and thermaal models
F Figure 5. 3D D spatio-th hermal mod del
998
Construction Research Congress 2012 © ASCE 2012
CONCLUSION AND FUTURE WORK With increasing efforts to improve building energy efficiency, the necessity of rapid energy modeling of existing buildings is on the rise to quickly and cost-effectively assess current conditions of building energy performances. To that end, we have presented a rapid 3D energy performance modeling of existing buildings using a novel digital and thermal image-based 3D reconstruction algorithm. The resulting 3D spatio-thermal model has a potential of being the basis of identifying potential candidates for building retrofit. Our initial results indicate the applicability of the 3D thermal point cloud models in providing an actual representation of thermal conditions within various interior spaces. Based on the thermal distribution in a given space, the 3D energy performance model could be used as a robust tool for retrofit analysis. It can ultimately help prioritize investments and minimize energy-loss cost in existing buildings. Streamlining modeling process would increase the implementation rate of building energy assessments and ultimately help meet energy-efficiency goals of existing buildings. The proposed work can reduce the time required to collect geometrical and energy performance data. Moreover, the underlying interior building 3D points can help practitioners to rapidly generate 3D geometrical models necessary for building energy performance simulations. In this context, the practitioners will spend less time for modeling and can focus more on the comparative analysis of possible retrofit alternatives. Thus, it can help provide further thinking about the retrofit alternatives. The presented work is part of a larger project in which the objective is to generate semantically-rich 3D building CAD models, map thermal values to different building elements, and measure the thermal comfort in various spaces. Future work includes generating semantically-rich CAD models from these point clouds, assigning thermal values as boundary conditions, and creating a Computational Fluid Dynamics (CFD) model to show the entirety of the actual energy performance of existing buildings. We also need to do more extensive experiments to reconstruct large scale models including ceilings and floors, and analyze the changes of energy performance over time in interior spaces. These results will be presented in a near future. AKNOWLEDGEMENTS Authors would like to thank Dawson Associates for their support with providing thermal cameras for initial experiments. REFERENCES Ahmad, M., and Culp, C.H. (2006). “Uncalibrated building energy simulation modeling results.” HVAC&R Research 12, 1141–1155. Architecture 2030. (2011). “Problem: The Building Sector.” , (Nov. 23, 2011). Autodesk. (2011). “Streamlining Energy Analysis of Existing Buildings with Rapid Energy Modeling.” 2011 Autodesk White Paper. Cho, Y. and Wang, C. (2011). “3D Thermal Modeling for Existing Buildings Using Hybrid LIDAR System.” Proc., 2011 ASCE Int Workshop on Computing in Civil Eng., Miami, F.
999
Construction Research Congress 2012 © ASCE 2012
Eads, L., Epperly, R., and Snell, J. (2000). “Thermography.” ASHRAE Journal, 42(3). Energy Information Administration (2010). “Annual Energy Review.” , (Nov. 23, 2011). Furukawa, Y. and Ponce, J. (2009). “Accurate, dense, and robust multi-view stereopsis.” IEEE Pattern Analysis & Machine Intel. Gonzalez-Aguilera, D., Rodriguez-Gonzalvez, P., and Gomez-Lahoz, J. (2010). "Camera and laser robust integration in engineering and architecture applications." Sensor Fusion and its Applications, ISBN 978-953-307-101-5. Golparvar-Fard, M., Peña-Mora, F., and Savarese, S. (2009). “D4AR- A 4Dimensional augmented reality model for automating construction progress data collection, processing and communication.” J. of ITCON, 14, 129-153. Golparvar-Fard, M., Bohn, J., Teizer, J., Savarese, S., and Peña-Mora, F. (2011). "Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques", Automation in Construction, 20(8), 1143-1155. Gould, K. and Hosey, L. (2009). “Ecology and Design: Ecological Literacy in Architecture Education.” AIA COTE/ Tide Foundation Award, 1-76. Hartley, R., and Zisserman, A. (2004). Multiple view geometry. Cambridge, UK: Cambridge University Press. Heo, Y., Choudhary, R., and Augenbroe, G. (2012). “Calibration of building energy models for retrofit analysis under uncertainty.” Energy and Buildings doi:10.1016/j.enbuild.2011.12.029 Key: citeulike:10217141. ImageModeler. (2011). “Create Photorealistic 3D Objects from Photographs.” Autodesk,, (Nov 2, 2011). Kiziltas, S, Akinci, B, Ergen, E, and Pingbo. T. (2008). “Technological assessment and process implications of field data capture technologies for construction and facility/infrastructure management.” J. of ITCON, 13, 134-154. Nüchter, A. (2012). Project ThermalMapper, , (Nov 1, 2011). Pelagottia, A., Del Mastio, A., Uccheddu, F., and Remondino, F. (2009). “Automated multispectral texture mapping of 3D models.” Proc., the 17th European Signal Conference (EUSIPCO 2009), Glasgow, Scotland. Photomodeler (2011) “Measuring and modeling the real world”, Eos Systems Inc. , (Nov 2, 2011). Roth, K., Westphalen, D., Feng, M., Llana, P., and Quartararo, L. (2005). “Energy impact of commercial building controls and performance diagnostics.” Technical Report, TIAX LCC, Cambridge, MA, 2005. Snavely, N., Seitz, S., and Szeliski, R. (2008). “Modeling the world from internet photo collections.” Int J Comp. Vis, 80(2) 189–210. Stockton., G (2010). “Using thermal mapping at the data center.” Proc., InfraMation. Tupper, K., Franconi, E., Chan, C., Hodgin, S., Buys, A., and Jenkins, M. (2011). “Building Energy Modeling: Industry-Wide Issues And Potential Solutions.” Proc. 12th Int Building Performance Simulation Association, Sydney. U.S. Department of Energy. (2010). “2010 U.S. DOE buildings energy databook.” , (Nov. 23, 2011).
1000
Rapid 3D Energy Performance Modeling of Existing Buildings using Thermal and Digital Imagery Youngjib Ham1 and Mani Golparvar-Fard2 1
Vecellio Pre-doctoral Fellow, Vecellio Construction Eng. and Mgmt., Via Dept. of Civil and Env. Eng., and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH (540) 235-6532; FAX (540) 231-7532; email: [email protected]
2
Assistant Professor, Vecellio Construction Eng. and Mgmt., Via Dept. of Civil and Env. Eng., and Myers-Lawson School of Construction, Virginia Tech, Blacksburg, VA; PH (540) 231-7255; FAX (540) 231-7532; email: [email protected]
ABSTRACT Rapid energy modeling is a streamlined process to capture, model, and analyze building energy performance. Timely assessment of existing building energy performance helps owners and facility managers to identify potential areas for better retrofit, and meet environmental and economic goals. Despite the increasing attention to building energy performance, the current process of energy data collection and modeling is time-consuming and requires certain level of expertise. In order to facilitate the process, this paper presents a new approach for actual energy performance modeling of existing buildings using digital and thermal imagery. First, using an image-based 3D reconstruction pipeline which consists of Structure-fromMotion, Multi-View Stereo, and Voxel Coloring/Labeling, the current geometrical condition of a building is captured. Subsequently, using a new 3D thermal modeling algorithm, a dense thermal point cloud model of the existing building is reconstructed in 3D. Finally, the reconstructed 3D building and thermal models are automatically superimposed within a single environment. Within the resulting 3D spatio-thermal point cloud model, temperature values can be queried and visualized at point level. The proposed method is validated on several rooms in an existing instructional facility. The underlying modeling process, and the potential benefits from converting digital and thermal imagery into ubiquitous sensors and reporters of building energy use are discussed in detail. INTRODUCTION The building sector uses about 40 percent of the total energy produced in the United States (DOE 2010). When looking at the life-cycle energy consumption of the U.S. buildings, nearly three quarters of produced electricity is consumed in operating buildings (DOE 2010). According to a recent report by the Energy Information Administration (EIA 2010), between 2010 and 2030, the increase in building sector’s energy consumption (7.16 Quadrillion Btu) will be more than those of the industrial and transportation sectors (4.05 and 3.36 Quadrillion Btu respectively). This energy is mainly produced from burning fossil fuels, making the building sector to be the largest producer of Greenhouse Gas emissions and the prime leading contributor to anthropogenic climate change (EIA 2010; Architecture2030 2010). Considering the significant energy consumption and the environmental impacts, government agencies are proposing incentive-based regulations to increase energy efficiency in residential and commercial buildings (e.g., 2011 Better Buildings Initiative). Currently several energy efficiency programs such as LEED and ENERGY STAR are becoming
991
Construction Research Congress 2012 © ASCE 2012
prevalent in the U.S. (DOE 2010). As a result of these mandatory and voluntary programs, a larger number of new buildings are being rigorously constructed to meet the requirement of these energy codes and efficiency programs. Nonetheless, a large number of existing buildings do not meet these standards, and their average age is rising. Retrofitting these buildings is important since they continuously undergo degradation over their service life. According to the U.S. Department of Energy (DOE 2010), 87 percent of overall residential buildings and 74 percent of total commercial buildings were built before 2000, and about 150 billion square feet of the existing buildings (i.e., roughly half of the entire building stock in the U.S.) will require renovations to meet the rigorous energy standards over the next 30 years (Gould and Hosey 2009). Faulty behaviors in existing buildings can alone account for 2~11 percent of the total building energy use (Roth et al 2005). Retrofitting existing buildings can potentially cut such excessive operational costs. Since any retrofit alternative directly translates into cost, building occupants are increasingly interested in identifying potential retrofit candidates of their buildings. Energy performance modeling of existing buildings is essentially an empirical exercise that relies on good observations to capture as-is building conditions. Once buildings are operating, the actual energy performance typically deviates from the baseline model, which is mainly due to the assumptions used during the modeling process. If the baseline energy model can closely represent the actual building energy performance, then it is more likely to provide a reliable source for the analysis of retrofit alternatives. To get a good match between the baseline models and the measured energy performance, modelers need to calibrate the baseline model parameters to exactly represent the actual building energy performance within an acceptable threshold. This is currently a widely accepted modeling process for analyzing existing buildings (Heo et al 2012; Ahmad and Culp 2006). Despite the effectiveness, several challenges can adversely impact the benefits: (1) Current energy modeling practices using existing building energy performance simulation tools (e.g., EnergyPlus, Ecotect, and eQuest) are timeconsuming and labor-intensive. First, building conditions are collected from the asplanned CAD models or by using traditional field surveying techniques (e.g., tape measurement or application of Total Stations). Subsequently, modelers should manually develop 3D energy models of existing buildings with the geometric information and various loads (e.g., weather data and occupancy schedules). Finally, extra time and effort for manual calibration of the model is needed to minimize the differences between the baseline models and the measured energy performance to a reasonably small level. Hence, the process of constructing models often requires weeks to months, and is therefore, often restricted to only high-budget projects (Autodesk 2011); (2) Existing energy models typically do not reflect a realistic picture of buildings as they are used. For example during modeling, all areas in a building are typically assumed to have similar insulation conditions, and each building element is assumed to have similar material conditions (e.g., degradation level). Hence, these tools primarily focus on comprehensive building operation (e.g., HVAC systems) or appliance-level energy use in a given thermal zone, and as a result cannot represent specific building areas in need for retrofit (e.g., construction defects or building
992
Construction Research Congress 2012 © ASCE 2012
degradation over time). Consequently, these approaches may not represent the accurate actual energy performance. (3) Current practices require certain level of skill and experience in energy modeling. According to a recent survey from energy modeling stakeholders (Tupper et al. 2011), one of the major bottlenecks of the current practice was identified as the difficulty in translating building geometrical and energy information into proper inputs for energy modeling. Overall, current labor-intensive and often inaccurate methods can delay the decision-making process by not delivering the necessary information to owner and stakeholders in a timely manner. Recent data collection technologies for building energy modeling (such as laser scanners and environmental sensor networks) are promising to eliminate nonvalue adding tasks within manual building and energy data collection process, and reduce time in modeling. However, they have severe practical challenges as they are not necessarily adaptable to all existing building conditions. For example, application of laser scanners for building data collection can be very costly, requires manual postprocessing of the data to remove noise and at least two experts to operate the device. The results can also suffer from mixed pixel phenomena within dynamic building environments (Golparvar-Fard et al. 2011, Kiziltas et al. 2008). Installation of environmental sensors for energy data collection can also be expensive and time consuming to set up for many existing buildings. To address these inefficiencies, this paper presents a new method for rapid energy performance modeling of existing buildings. The proposed method leverages recent image-based 3D modeling approaches as well as thermal cameras. By using inexpensive and often existing digital images in addition to thermal imagery, our proposed method automatically generates 3D spatio-thermal models, and enables owners and facility managers to quickly get the actual energy performance data for their existing buildings. In the following sections, first the state of knowledge in the areas of 3D thermal modeling is briefly overviewed. Next, the research objectives and methodologies are presented. Finally, we present preliminary results as a proof of concept for the proposed method, and conclude with a discussion on the potential benefits and limitations. BACKGROUND AND RELATED WORK Thermography - Thermography is defined as detecting and measuring variations in heat emitted by an object and transforming them into visible images (Eads et al. 2000). In terms of building energy performance modeling, thermography is a robust tool in recording, analyzing, and reporting actual energy performance of existing buildings. Thermal images from buildings are directly influenced by the building energy performance such as energy transfer through building elements (e.g., thermal bridges) or space heating and cooling energy related to HVAC systems. Currently the majority of applications of thermal images (e.g., thermographic inspections) are based on 2D images of the scene. Since these images can only show what is in their field of view and are not geo-tagged, they are not sufficient to provide a whole picture of energy performance in a given space. To advance both the measurement and the interpretation of energy performance, there is a need to consider 3D thermal modeling. A 3D model containing thermal information enables to recognize how
993
Construction Research Congress 2012 © ASCE 2012
temperature values are spatially distributed in a given space in a holistic manner. 3D thermal modeling – A number of previous research studies in both computer science as well as building construction domains have proposed several new methods for 3D thermal modeling. Stockton (2010) suggested a 3D thermography monitoring system to manage power consumption, cooling, and IT operations in a data center. The results show the effectiveness of 3D thermography, yet due to the manual process involved in creating image mosaics and performing texture-mappings, the proposed approach is time-consuming and labor-intensive. Cho and Wang (2011) introduced a 3D thermal modeling using a LIDAR scanner and a thermal camera. The 3D geometrical information of existing building envelopes are first collected using the laser scanner, and then temperatures values are mapped to building exterior surfaces using the thermal camera. Similarly, Nüchter (2012) proposed constructing 3D thermal models of indoor environments using a robot equipped with a laser scanner and a thermal camera. To collect the building and energy data, these two studies require a large and dedicated equipment to support both a laser scanner and a thermal camera at the same time. However, due to complex and confined indoor environment, this approach may not be easily applicable to interior spaces. Moreover, laser scanning does not provide any semantic information of the appearance of the building elements which is attributed to the lack of features in Cartesian point clouds generated by laser scanners. Although several previous studies from computer science (Pelagottia et al. 2009; Gonzalez-Aguilera et al. 2010) generally yield accurate 3D thermal models, yet they only focused on confined isolated objects or single image pairs using multiple thermal cameras to build 3D thermal models (e.g., a combination of one digital camera and two thermal cameras or stereo thermal camera systems). These approaches would require building auditors to acquire more thermal cameras which comes at a higher price and may not significantly add value. OVERVIEW OF THE PROPOSED 3D ENERGY MODELING Our work explores the application of 2D digital and thermal imagery collected with a single thermal camera. The proposed method models current building geometrical information and visualizes actual energy performance at the level of both building elements and various spaces within a single virtual 3D environment. Given a collection of unordered and uncalibrated digital and thermal imagery, our goal is to automatically reconstruct a 3D model and assign temperature values to each point within the 3D reconstructed scene. In our approach, the digital images are capturing the as-is condition of existing buildings, and the thermal imagery are capturing the actual energy performance. Since our proposed method automatically generates 3D actual energy performance models with a large number of unordered digital and thermal images, it has the benefit of streamlining the modeling process. In addition, due to the portability of one single thermal camera (integrated with a digital camera) and its mobility in an indoor environment, it is suitable for building interior surveys. Furthermore, our work makes it possible to quickly and cost-effectively understand the actual thermal performance by help of 3D reconstructed semantically-rich model. Thus, the word ‘rapid’ in this study not only refers to the modeling process itself (i.e., which is streamlined and does not require expertise and manual process), but also the quick screening method to identify potential areas required for more detailed energy
994
Construction Research Congress 2012 © ASCE 2012
annalysis. In the followiing, the research metthodology aand validattion are furrther deetailed. Severaal studies suuch as (Auttodesk 20111) have alreeady provenn the benefitts of im mage-based 3D reconstrruction com mpared to coonventionall surveys orr laser scannning appproaches (e.g., mannual or suupervised approachess similar to Autodeesk’s Im mageModeleer (2011) oor PhotoMoodeler (2011)). Figure 1 shows an example of theermal and digital d imagges. Both off these sourcces of imagges were capptured usingg the sam me camera in Figure 1a. Imagess labeled with ‘b’ in F Figure 1 show the therrmal chharacteristicss of the sam me areas captured in ‘c’..
F Figure 1. Th hermal and d digital imaages in exissting buildin ngs Figurre 2 presentts an overviiew of data and processs in our prooposed metthod. Thhe followingg describes eeach step w within our appproach:
F Figure 2. O Overview off 3D energyy performan nce modelin ng Im mage-based d 3D recon nstruction – Our method initiaally takes unordered and unncalibrated digital im mages and implemennts a streaamlined im mage-based 3D recconstructionn algorithm which conssists of Struucture-from--Motion (SffM) (GolparrvarFaard et al. 2009, Snaveely et al. 20008), Multii-View Sterreo (MVS) (Furukawa and Poonce 2009),, and Voxell Coloring/L Labeling (V VCL) (Golpparvar-Fard et al. 20111) to creeate dense 3D buildingg point cloud models of the scenne. To geneerate 3D spatiotheermal modeels, thermal and digitall images need to be coo-registered to map therrmal vaalues to the reconstructe r ed 3D scenee. To that ennd, our initiaal approach was to form m the Eppipolar geom metry (Harttley and Zissserman 20004) betweenn every paiir of digital and theermal images and com mpute the Fuundamental matrix betw ween each ppair. In ordeer to auutomate thiss process forr all pairs, w we conductted various exhaustive experimentts on
995
Construction Research Congress 2012 © ASCE 2012
996
thee state-of-tthe-art invvariant featture annd corner detectionn algorithhms inccluding SIF FT, ASIFT, SURF, MS SR, annd Harris Corner detection to invvestigate w whether featuure points can c Figu ure 3. Therm mal cameraa calibration n bee detected and a matchedd across thhese paairs. In almoost all casess, these algoorithms worked poorlyy or did nott work at alll (in moost cases reesulted in deetection of lless than 100 correspondding points)). This is duue to typpical low rresolution of o thermal iimages, lackk of distincct features, and signifiicant raddial distortiion in therm mal imageryy. Thermal cameras usse gradient color codinng to caapture therm mal performance whichh smoothen all surface intensities, and as a reesult theere is no disstinct featurre that can bbe detectedd. To solve tthis challengge, we com mpute thee intrinsic pparameters oof the therm mal camera as a follows: Th hermal cam mera interrnal calibrration – Inn order to calibrate ddigital cameeras, caalibration rigs, in form m of checkkerboard, arre typicallyy used. How wever, the low ressolution thhermal cam meras whichh can onlyy detect thhermal diffferences caannot disstinctly disttinguish inteersecting coorners of a ccalibrating checkerboar c rd. Hence inn our woork, we creeate a therm mal calibrattion rig (5550mm x 7000mm) usingg 42 small L LED ligghts locatedd on the inteersections of the conveentional checkerboard ((See Figure 3a). Prrior to calibbration, the temperaturee range for thermal maap color coding is fixeed to maake sure each gradientt of color is correspondding to an aabsolute tem mperature vaalue. Onnce the lighhts are on, thhey will geenerate heatss, and the thhermal cam mera which have h theermal sensitivity of 0.005°C can eaasily distincct these corrners (See F Figure 3b ~ 3e). Thhrough this process, thee thermal caamera is intternally calibbrated (i.e. the focal length ( ) and raadial distorttion parameeters ( and oof the therm mal cameraa are caalculated). N Next, assumiing a pinhole camera m model, the loocal coordinnates of therrmal im magery denooted as aare undistortted by usingg: , , (1) In thiis equation, raddial distortioon:
is a fuunction thatt computes a scaling facctor to undoo the 1.0
‖ ‖
‖ ‖
(2)
This is an imporrtant step in our process, as almostt all thermal cameras cause siggnificant raddial distortion due to thheir wide-anngle lens whhich is usedd to increasee the fieeld of view and minim mize the tim me required for data coollection. Foor most therrmal caameras, this process onlly needs to bbe done oncce and does not need too be repeatedd for diffferent data collections since the thhermal cameera lens is fiixed. Deense 3D th hermal modeling – Next, N we exxtract the eextrinsic parrameters off the theermal cameeras (i.e., cam mera locatioon ( ) and orientation o ( )) from thhe results off the SffM step connducted on ccorrespondiing digital images taken in the sam me locationn and orientation with w the therrmal camerra (this appproach is vaalid for moost new therrmal mages caameras as tthey have the abilityy to capturre both diggital and tthermal im sim multaneouslly). Based oon the intrinnsic and exttrinsic cameera parametters, the therrmal caamera projecction matrixx ( for ppreviously undistorted thermal im mages is forrmed ussing:
Construction Research Congress 2012 © ASCE 2012
0 0 0
0
997
/2 /2 1
(3)
wherein and are the width and the height of the thermal images. The undistorted thermal images along with their camera projection matrices are fed into the dense 3D reconstruction steps (MVS and VCL) and yield a dense 3D thermal model. Integrated visualization of digital and thermal imagery and models – In our work, we have modified the D4AR – 4D Augmented Reality Model viewer (Golparvar-Fard et al. 2009) to enable joint visualization of digital and thermal 3D point cloud models and imagery. The resulting 3D spatio-thermal models are represented using the following data structures: , ,.., which consist of a 3D location and a A set of digital points Ƥ color combined from one of the digital images where the point was observed. , ,.., A set of digital cameras Ƈ which consist of a 3D location and a A set of thermal points Ƥ , ,.., color combined from one of the thermal images where the point was observed. A set of thermal cameras Ƈ , ,.., A mapping between the points and the cameras that observe them. EXPERIMENT RESULTS Table 1. Camera Technical Data AND DISCUSSIONS Camera Technical Items Characteristics Hardware – The digital and Built-in Digital Camera 2048 × 1536 pixels thermal images were captured Thermal Image Resolution 320 × 240 pixels using an E60 thermal camera Thermal Sensitivity 0.05°C from FLIR Systems AB, Thermal Measurement Accuracy ±3.6°F(±2°C) which has a built-in digital Digital Camera Field of View (FOV) 53° × 41° 25° × 19° camera. Table 1 reports the Thermal Camera FOV Minimum Focus Distance 1.31 ft.(0.4m) camera specifications. Data Collection – To test the proposed method, we performed experiments in an office room of an existing instructional facility on campus of Virginia Tech. The visual data were collected during a morning and under natural daylight settings. Results – Table 2 shows the results of our experiment. Figure 4a, b, c, and f represent the dense 3D reconstructed scenes from digital and thermal images, and illustrate one of the locations and orientations of the camera registered in a virtual 3D environment. Once a registered camera is visited (Figure 4d and g), the frustum of the camera is automatically texture-mapped with a full resolution of the image used to capture it (Figure 4e and h). Thus, the user can visually and interactively acquire information related to thermal performance Table 2. Experiment Outcomes and building condition of the Results Values 429 given area from both digital The # of Digital Images 429 and thermal imagery. Figure The # of Thermal Images 2,064,662 points 5a shows the result of Density of Thermal 3D Point Cloud Density of Building 3D Point Cloud 8,488,888 points superimposition of a 3D Computation Time ~ 6 hours building model over a 3D Benchmarked on an Intel(R) Core(TM) i7 960 with 24GBs of RAM
Construction Research Congress 2012 © ASCE 2012
theermal modeel based on the same coordinate c ssystem. A 33D spatio-thhermal moddel is auutomaticallyy texture-maapped with a full resoluution of therrmal image (Figure 5b)). 3D sem mantically-rrich reconsttructed moddels from diggital imagess allow to beetter undersstand hoow temperatture values are spatiallly distributted with seemantic infoormation off the givven space ffor a givenn point of tiime. More experimenttal results ccan be founnd at ww ww.raamac..cee.vt.edu/eepar.
Figure 4. Dense 3D D building and thermaal models
F Figure 5. 3D D spatio-th hermal mod del
998
Construction Research Congress 2012 © ASCE 2012
CONCLUSION AND FUTURE WORK With increasing efforts to improve building energy efficiency, the necessity of rapid energy modeling of existing buildings is on the rise to quickly and cost-effectively assess current conditions of building energy performances. To that end, we have presented a rapid 3D energy performance modeling of existing buildings using a novel digital and thermal image-based 3D reconstruction algorithm. The resulting 3D spatio-thermal model has a potential of being the basis of identifying potential candidates for building retrofit. Our initial results indicate the applicability of the 3D thermal point cloud models in providing an actual representation of thermal conditions within various interior spaces. Based on the thermal distribution in a given space, the 3D energy performance model could be used as a robust tool for retrofit analysis. It can ultimately help prioritize investments and minimize energy-loss cost in existing buildings. Streamlining modeling process would increase the implementation rate of building energy assessments and ultimately help meet energy-efficiency goals of existing buildings. The proposed work can reduce the time required to collect geometrical and energy performance data. Moreover, the underlying interior building 3D points can help practitioners to rapidly generate 3D geometrical models necessary for building energy performance simulations. In this context, the practitioners will spend less time for modeling and can focus more on the comparative analysis of possible retrofit alternatives. Thus, it can help provide further thinking about the retrofit alternatives. The presented work is part of a larger project in which the objective is to generate semantically-rich 3D building CAD models, map thermal values to different building elements, and measure the thermal comfort in various spaces. Future work includes generating semantically-rich CAD models from these point clouds, assigning thermal values as boundary conditions, and creating a Computational Fluid Dynamics (CFD) model to show the entirety of the actual energy performance of existing buildings. We also need to do more extensive experiments to reconstruct large scale models including ceilings and floors, and analyze the changes of energy performance over time in interior spaces. These results will be presented in a near future. AKNOWLEDGEMENTS Authors would like to thank Dawson Associates for their support with providing thermal cameras for initial experiments. REFERENCES Ahmad, M., and Culp, C.H. (2006). “Uncalibrated building energy simulation modeling results.” HVAC&R Research 12, 1141–1155. Architecture 2030. (2011). “Problem: The Building Sector.” , (Nov. 23, 2011). Autodesk. (2011). “Streamlining Energy Analysis of Existing Buildings with Rapid Energy Modeling.” 2011 Autodesk White Paper. Cho, Y. and Wang, C. (2011). “3D Thermal Modeling for Existing Buildings Using Hybrid LIDAR System.” Proc., 2011 ASCE Int Workshop on Computing in Civil Eng., Miami, F.
999
Construction Research Congress 2012 © ASCE 2012
Eads, L., Epperly, R., and Snell, J. (2000). “Thermography.” ASHRAE Journal, 42(3). Energy Information Administration (2010). “Annual Energy Review.” , (Nov. 23, 2011). Furukawa, Y. and Ponce, J. (2009). “Accurate, dense, and robust multi-view stereopsis.” IEEE Pattern Analysis & Machine Intel. Gonzalez-Aguilera, D., Rodriguez-Gonzalvez, P., and Gomez-Lahoz, J. (2010). "Camera and laser robust integration in engineering and architecture applications." Sensor Fusion and its Applications, ISBN 978-953-307-101-5. Golparvar-Fard, M., Peña-Mora, F., and Savarese, S. (2009). “D4AR- A 4Dimensional augmented reality model for automating construction progress data collection, processing and communication.” J. of ITCON, 14, 129-153. Golparvar-Fard, M., Bohn, J., Teizer, J., Savarese, S., and Peña-Mora, F. (2011). "Evaluation of image-based modeling and laser scanning accuracy for emerging automated performance monitoring techniques", Automation in Construction, 20(8), 1143-1155. Gould, K. and Hosey, L. (2009). “Ecology and Design: Ecological Literacy in Architecture Education.” AIA COTE/ Tide Foundation Award, 1-76. Hartley, R., and Zisserman, A. (2004). Multiple view geometry. Cambridge, UK: Cambridge University Press. Heo, Y., Choudhary, R., and Augenbroe, G. (2012). “Calibration of building energy models for retrofit analysis under uncertainty.” Energy and Buildings doi:10.1016/j.enbuild.2011.12.029 Key: citeulike:10217141. ImageModeler. (2011). “Create Photorealistic 3D Objects from Photographs.” Autodesk,, (Nov 2, 2011). Kiziltas, S, Akinci, B, Ergen, E, and Pingbo. T. (2008). “Technological assessment and process implications of field data capture technologies for construction and facility/infrastructure management.” J. of ITCON, 13, 134-154. Nüchter, A. (2012). Project ThermalMapper, , (Nov 1, 2011). Pelagottia, A., Del Mastio, A., Uccheddu, F., and Remondino, F. (2009). “Automated multispectral texture mapping of 3D models.” Proc., the 17th European Signal Conference (EUSIPCO 2009), Glasgow, Scotland. Photomodeler (2011) “Measuring and modeling the real world”, Eos Systems Inc. , (Nov 2, 2011). Roth, K., Westphalen, D., Feng, M., Llana, P., and Quartararo, L. (2005). “Energy impact of commercial building controls and performance diagnostics.” Technical Report, TIAX LCC, Cambridge, MA, 2005. Snavely, N., Seitz, S., and Szeliski, R. (2008). “Modeling the world from internet photo collections.” Int J Comp. Vis, 80(2) 189–210. Stockton., G (2010). “Using thermal mapping at the data center.” Proc., InfraMation. Tupper, K., Franconi, E., Chan, C., Hodgin, S., Buys, A., and Jenkins, M. (2011). “Building Energy Modeling: Industry-Wide Issues And Potential Solutions.” Proc. 12th Int Building Performance Simulation Association, Sydney. U.S. Department of Energy. (2010). “2010 U.S. DOE buildings energy databook.” , (Nov. 23, 2011).
1000
