Water use efficiency (WUE), defined as the ratio of gross primary productivity and evapotranspiration at the ecosystem scale, is a critical variable linking the carbon and water cycles. Incorporating a dependency on vapor pressure deficit, apparent underlying WUE (uWUE) provides a better indicator of how terrestrial ecosystems respond to environmental changes than other WUE formulations. Here we used 20th century simulations from four terrestrial biosphere models to develop a novel variance decomposition method. With this method, we attributed variations in apparent uWUE to both the trend and interannual variation of environmental drivers. The secular increase in atmospheric CO2 explained a clear majority of total variation (66 ± 32%: mean ± one standard deviation), followed by positive trends in nitrogen deposition and climate, as well as a negative trend in land use change. In contrast, interannual variation was mostly driven by interannual climate variability. To analyze the mechanism of the CO2 effect, we partitioned the apparent uWUE into the transpiration ratio (transpiration over evapotranspiration) and potential uWUE. The relative increase in potential uWUE parallels that of CO2, but this direct CO2 effect was offset by 20 ± 4% by changes in ecosystem structure, that is, leaf area index for different vegetation types. However, the decrease in transpiration due to stomatal closure with rising CO2 was reduced by 84% by an increase in leaf area index, resulting in small changes in the transpiration ratio. CO2 concentration thus plays a dominant role in driving apparent uWUE variations over time, but its role differs for the two constituent components: potential uWUE and transpiration.