Review of “A Comprehensive Analysis of Groundwater Quality in the Barnett Shale Region”

When a reporter asked for my opinion about a recently released study of groundwater in the Barnett Shale area (Hildenbrand et al., University of Texas-Arlington), I was quoted as saying that, despite its title, a comprehensive analysis of groundwater quality in the Barnett Shale was not achieved in this paper. In response to that comment, one of the authors posed the rhetorical question to another reporter: “We tested over 500 water wells; what does he want, 1,000?” The authors then sent me a note asking what I considered to be comprehensive.

Below, I will lay out my answer to that question, which is far more complicated than a simple numbers game to be played within the news media.  To summarize, the UTA paper suffers from a number of problems including

•Inadequate disclosure of data collection and testing quality assurance/quality control. 

•No presentation of the data or the test results that would allow other researchers to study and attempt to reproduce the results.

•Implicating unconventional oil and gas extraction, or implication-by-raw-possibility, without adequate evidence.

•No baseline data, i.e. water quality data before unconventional oil and gas extraction began.

•No serious consideration of the many other possible causes of contaminated well water other than unconventional oil and gas extraction.

Due to these problems, the UTA paper does not present evidence that shows any relationship between water quality and unconventional oil and gas extraction in the Barnett Shale region, yet the researchers manage to presume a linkage throughout the paper. 

A scientifically objective and comprehensive analysis isn’t fully defined by the number of wells sampled. It begins with a null hypothesis, followed by a rigorous attempt to disprove it with extensive investigation. In this case, it would mean an investigation of all potential sources of groundwater contamination in the region, or at least a significant number of sources.  In the parlance used by these researchers, the null hypothesis to test would be: “groundwater in the Barnett Shale area has not been contaminated by unconventional oil and gas extraction (UOG).”

The next step would then be to test a sufficient number of water wells to determine if a statistically significant number of water wells show statistically significant levels of contamination.  Beyond just the number of wells, though, the wells would also need to be normally distributed throughout the study area.  A comprehensive analysis of the test results would then attempt to disprove the null hypotheses with an exhaustive analysis of the conceivable causes of contaminated groundwater – not just UOG.

In addition to ensuring the sample population is significant, the quality of the data must be above reproach to support the claims made in the paper.  In Hildenbrand et al., readers are not privy to the Quality Assurance/Quality Control procedures, sampling protocol, analytic methods, or analytic results of the samples collected. This is not to suggest automatically that the procedures employed in Hildenbrand et al. are flawed, but an inadequate discussion of such procedures raises red flags with any research. The bane of any research is bad data, and simply taking the word of the authors is not appropriate scientific scrutiny.

Additionally, a cornerstone of the scientific method is reproducibility. The tests conducted (and conclusions drawn) by a researcher should be able to be replicated by another through a reasonable evaluation of the same or similar data set within the same study area.  Without access to the data (analytics and well data) the reader is forced to rely on the opinion of the paper’s authors – and, in this case specifically, the news media’s skewed interpretation of the report.

It appears to me that this paper started with the assumption that groundwater in the Barnett Shale area had been contaminated by UOG, and – with few exceptions – other potential sources of contamination were either not investigated sufficiently or overlooked. Some sources, such as agricultural activity and urbanization, were occasionally mentioned, but they were rejected in the paper with little analysis. 

To be fair, the paper concludes with this statement:
“The detection of numerous volatile organic compounds in aquifers above the Barnett shale does not necessarily implicate unconventional UOG extraction as the source of contamination...” 

However, by the time this statement is made at the very end, readers have been pummeled by implication after implication that UOG activities are likely, possibly, or could be responsible for the observed groundwater contamination.
For example, in one section the authors state, “High levels of chloride and bromide can be an indication of anthropogenic contamination, one possibility being UOG activity, a result of groundwater mixing with produced water from the shale formation” (p. 303-305; emphasis added).  But then the authors then spend the next two pages showing why a surface source is the likely culprit, not UOG activity:  “Additionally, a composite variable derived from a PCA of chloride, bromide, and nitrate … was negatively correlated with depth of groundwater well (r = -0.34, p = 0.002), suggesting a surface source may contribute to observed concentrations of these contaminants” (p. 312-317; emphasis added).

With so much effort put into showing that the likely cause of contamination was something on the surface, the authors should have clearly stated that UOG activity was not the likely cause.  But by stating that UOG activity was a possible cause of contamination, the authors were guilty of editorializing (i.e. implicating UOG activity) instead of adhering to the rigorous standards as set forth in the scientific method. 

While not entirely the fault of the research team, this implication-by-raw-possibility opened the door for political groups to use the study in their campaign against drilling. Earthworks, for example, sent out a press release that captured many reporters’ attentions, arguing that the study found “widespread groundwater pollution from fracking chemicals.”  Even though the Earthworks press release said that the authors of the paper did not endorse their press release, the authors also did not publicly denounce the press release. 

The resulting news coverage of the report focused on the supposed linkage between UOG and groundwater contamination, and there’s no indication that the UTA research team attempted to correct the record. Was the news coverage of the report too valuable to risk a public request for corrections? Was the goal to capture news attention, or to investigate groundwater quality in parts of North Texas?

Without baseline data or a comprehensive exploration of other potential sources of groundwater contamination, this paper offers little insight into why so many water wells in North Texas contain the compounds identified in Hildenbrand et al.  The researchers not only did not have baseline data, but also admitted that their water samples were not random, the latter of which raises significant questions about the objectivity of the entire enterprise. As the authors of the paper themselves observed:“[T]his opportunistic and necessarily biased sampling hindered our ability to make meaningful inferences regarding levels of contamination as a function of distance from nearest UOG well for several reasons…”

There have been plenty of other investigations of water quality in the region that identified other industrial sources as possible culprits of contamination. For example, a large number of water wells that tap into the Trinity aquifer have long been contaminated, probably from a wide range of sources including “fertilizers, septic tank effluent, municipal sewage, animal feedlots, decaying vegetation and atmospheric deposition,” as noted by P. E. Hudak in 2012 based on water well samples collected in 2007. Other potential sources of valid historic background water quality data include, but are not limited to, the Joint Groundwater Monitoring and Contamination Report 2013, SFR-056/13 (and previous volumes – TCEQ), the Texas Water Development Board Report 269 by Phillip L. Nordstrom (April 1982), and data collected and managed by local groundwater conservation districts.

Indeed, groundwater contamination from surface activities that predate UOG is extensive in the Barnett Shale area.  One operator in the Barnett Shale, Beacon E&P, made these comments: At our One Prime drill site roughly a mile north and east of 360 at I-30, however, before even permitting the pad for construction and drilling, we found we needed to get an IOP (Innocent Operator Program) Certification in place with the TCEQ because there were documented contaminants existing in the shallow groundwater.  These had been tied by several earlier studies to an aircraft parts manufacturing facility formerly nearby.

That plume of chlorinated hydrocarbons and BTEX and VOCs was long known by the environmental community to be extensive in the area, and many similar plumes apparently exist around Carswell AFB and throughout Tarrant County and beyond (I was told) that are likely due to in part to the industrial activity in the region since before World War II. 

We were glad to find out about it, and get our IOP, because that gave us the opportunity to set additional shallow protection (casing) strings before drilling our wells and we were able to prevent any further cross contamination of other freshwater horizons.

There is always more research that needs to be done.  All research should be considered one more step in the pursuit of science, but my opinion is that the UTA paper could have done a better job of exploring all potential sources of groundwater contamination, rather than focusing on and implicating UOG as the most likely source.

Thus, my observation that this was not an adequate or comprehensive analysis has not changed. It does not mean that the study is without value, but it is also difficult to endorse a report using testing that cannot be replicated, a non-random and admittedly “biased” data set, and a lack of appreciation for the many other sources of water contamination that have affected North Texans for decades. Due to these problems, the paper does not present evidence that shows any relationship between water quality and unconventional oil and gas extraction in the Barnett Shale region, yet the researchers manage to presume a linkage throughout the paper.