September 7, 2017

POINT OF CONTACT

Principle investigator:
     Richard Eckman
     NOAA Air Resources Laboratory Field Research Division
     1750 Foote Dr.
     Idaho Falls, ID 83402
     Richard.Eckman@noaa.gov
     (208) 526-2740

README.TXT - Description of Data Files from ARLFRD Sonic Anemometers

Sonic anemometers were deployed during PSB2 to measure the turbulence 
field driving the tracer dispersion in the horizontal and vertical. The 
sonics measured the turbulence by taking high frequency (10 Hz) measurements 
of the 3-d wind field and temperature (u, v, w, t).  A 3-d sonic anemometer 
sample consisted of transmitting sound back and forth across the 
measurement volume of the anemometer.  The delay between transmission and 
receipt of a sound pulse in both directions along the 3 axes of the anemometer 
yields wind speed and direction in 3 dimensions. Virtual temperature was also 
derived from the speed of sound across the sonic sampling volume.

For measurements of the vertical turbulence profile on GRI, ARLFRD deployed 
six 3-d sonic anemometers during the study. These were R. M. Young Model 
81000 Ultrasonic anemometers installed at 2, 3.7, 9, 16.5, 30, and 60 m agl. 
The 2 and 3.7 m sonics were mounted on tripods near the tower. The sonics 
from 9 m agl upwards were mounted on booms extending the same distance from 
the tower on a 335 deg azimuth. Power was supplied to the sonics and all other 
instrumentation on GRI by gel cell batteries. The batteries on GRI were 
continuously charged by AC line power. One Gill Windmaster Pro sonic 
anemometer (G2) was installed on a tripod at 3.1 m agl on the 1000 m arc at 
150 deg azimuth.
 
The ARLFRD sonic data were continuously recorded for the duration of the 
experimental period at 10 Hz on a compact flash card inserted into an 
Acumen Serial Data Collection Bridge. The data bridge was configured manually 
with a laptop computer with the sonic designation at the start of its 
filename (e.g., R1, G2). A GPS unit was also used to verify, and synchronize if 
needed, the correct time in the data bridge.  The sonic data were recorded in an 
ASCII text file. The compact flash cards were gathered and returned to ARLFRD for 
processing and data archival at regular intervals during the experimental period.  

Quality Control  	

Once the flash cards were returned to ARLFRD, the data was uploaded onto the 
network for processing. The 10 Hz data was parsed into both 10-minute and 
30-minute files containing roughly 6000 and 18,000 observations, respectively, 
for the entire PSB2 experimental period. Means, variances, covariances, and other 
calculations were made on these 10 and 30-minute data files.  	

The 10-minute averaging time was selected to match with the 10-minute averaging 
period of the SF6 tracer sampling period. The longer 30-minute averaging period 
included for some of the measurements in the database better accounts for
nonstationarity effects in the flow and provides a more robust statistical 
measurement of the turbulent fluxes. If necessary, the data was rotated into 
the correct meteorological coordinate system prior to processing (60 degrees, 
Gill sonic anemometer only).    	

Data collected from the seven ARLFRD sonic anemometers were subjected to a 
comprehensive quality control and processing software package based upon the 
schemes detailed in Vickers and Mahrt (1997). That package included spike 
detection, 2-coordinate rotation to zero out the mean vertical and crosswind 
wind speeds and calculate the streamwise mean wind, range checks, amplitude 
resolution tests, dropout tests, Haar transform stationarity (discontinuity) 
tests, checks for excessive skewness and kurtosis, tests for relative systematic 
flux sampling error (RSET), tests for random flux sampling error (RFET), tests 
for flux variations associated with mesoscale motions (RNT), and tests for 
alongwind relative nonstationarity (RNU), crosswind relative nonstationarity 
(RNV), and vector wind relative nonstationarity (RNS).   	

The most common problem with sonic anemometer measurements is spiking in which 
large, random, very brief, and infrequent electronic signal noise is recorded. 
Spikes were detected based upon the criteria of Vickers and Mahrt (1997), with 
slight modification. This entailed identifying 3 or less consecutive points 
exceeding the mean  a multiple of the standard deviation for a 5-minute 
(3000 point) moving average. The thresholds used were 4.0 standard deviations 
for u, v, w, and t. For w, the standard deviation threshold was adjusted to 4.5 
if it was nighttime with temperatures below freezing. This was done to account 
for nocturnal periods often characterized by low and very intermittent turbulence. 
Spikes were replaced recursively by the mean of the nearest non-spike values on 
either side of the spike. The spike replacement routine was repeated for up to 
11 passes through the record or until no spikes were detected. The threshold 
increased by 0.1 with each pass. The quality control data files provide information 
on both the total number of passes through the record and the cumulative number of 
spikes detected in all passes. If the total number of spikes detected for any 
channel exceeded 0.5% of the total record on any pass, the record was flagged 
accordingly. Final calculations were done using the despiked output files. These 
calculations included both the uncorrected and corrected mean and flux quantities 
as well as a suite of quality control parameters.   	

The resulting despiked sonic anemometer data sets were plotted and reviewed by 
the data analyst for consistency and accuracy by comparing results with other 
measurements for the duration of each IOP plus one hour before and after each 
IOP.  This included the following comparisons:
	Time series comparisons in the horizontal between all wind speed and 
direction measurements in the study area, where available, at 2 m; at 10 m; 
at 15 m; at 30 m; at 45 m; and at 60 m. These comparisons included the sonic 
anemometers, cup anemometer and wind vanes at GRI and COC, and the ASC sodar. 
Heights were compared if they were close. For example, 2 m cup and vane results 
at GRI and COC were compared with sonic results at 3 m (e.g., G2, EC1) and 40 m 
results for the ASC sodar were compared to 45 m anemometer results at GRI.
	Vertical profile comparisons between sonic anemometer measurements and 
cup anemometer and wind vane measurements on GRI.

The cup anemometers and wind vanes on GRI were calibrated to rigorous standards. 
Like the other meteorological towers in the NOAA/INL Mesonet, a detailed and 
comprehensive data quality assurance program is performed on GRI on a routine 
basis. The instrumentation, quality control, calibration, and maintenance 
procedures at GRI meet the generally accepted requirements and guidelines set 
out in DOE (2004, 2005), ANSI/ANS-3.11 (2005), and ANSI/ANS-3.2 (2006).  To 
help follow these guidelines, the quality assurance program uses an excellent 
set of software tools to display trended meteorological data. This enhances 
the data quality evaluations and makes them more efficient. The quality control 
program consisted of both manual and automated processes. Every 5-minute period 
for each parameter was plotted for missing or spiked data. Data were also 
screened for electronic noise, non-working aspirators that affect air temperature 
and relative humidity values, orientation errors in the wind direction, stalled 
wind sensors, rime icing that degrade wind speeds, and other erroneous values 
caused by maintenance, bird droppings, etc. Plotting the data allows the 
meteorologist to identify and flag any of the problems in the database and, if 
needed, a technician is notified to quickly fix the problem. Calibrations of all 
instrumentation are completed on a semi-annual basis.

The results for these comparisons are included in sets of graphs in each IOP 
summary (see Summary of Individual IOPs section).

The plots of the new data sets were reviewed and verified by a second analyst. 
If any problems or errors were discovered, the two analysts had to agree upon 
and implement a resolution. An error in the G2 sonic anemometer was identified 
and corrected between IOPs 2 and 3. It was found that the G2 Acumen had been 
initially programmed in MDT whereas all the other instrumentation was in MST. 
Wind speeds at the 2, 10, and 30 m heights for COC were also found to be 
anomalously high in the initial review for IOP4. It was found that the maximum 
5-minute gusts instead of the mean 5-minute wind speeds had been incorporated 
into the review plots for that IOP. This was corrected and then good agreement 
was found. The protocols used to generate the data sets described below were 
derived from Vickers and Mahrt (1997).

Data File Formats

There are 7 ARLFRD files containing 10-min average processed data, six for the 
sonic anemometers on GRI and one for the G2 sonic for each IOP. The data files 
are summaries of the measurements and calculated quantities for each 24 h period 
encompassing an IOP test. Each file is in CSV format. The data filenames are 
specified as PSB2data_FRD_IOPx_XX_yy_##min.csv where x is the IOP test number, 
XX is the identity of the sonic anemometer specified in Table 16 (e.g., G2, R1), 
yy is the agl measurement height, and ## is the averaging period (10 or 30 
minutes). There are corresponding quality control 10-min average files with the 
filenames PSB2qc_FRD_IOPx_XX_yy_##min.csv. The quality control files contain a 
listing of quality control parameter values and flags for each period. Data from 
the G2 sonic was unavailable for IOP4.

There are a total of 14 FRD files containing 30-min average processed data. Seven 
of these are data files and seven are qc files, one pair for each sonic 
anemometer. Each file contains two blocks of data, one covering the daytime IOPs 
and one covering the nighttime IOPs.

Some data have been automatically flagged out with '-9999' due to flags set in 
the 'qc' files, columns 4-7 and 24-27. These are for excessive number of spikes 
or min/max values exceeding certain thresholds, respectively. Temporal gaps 
with missing data are also flagged '-9999'. The corresponding 'data' and 'qc' 
files contain matching records by row. The temperatures reported are virtual 
sonic temperatures.

The column header designations for the data summary files are:

1. XXMMDDYYHRMN, XX is the identity of the sonic anemometer, MM is 
                the month, DD is the day, YY is the year, and HRMN is the 
		starting hour and minute of the 10 or 30 minute averaging period 
		for that row			
2. KNT    	Data points in interval
3. VECWD    	Vector Wind Direction (despiked)   [degrees azimuth]
4. SCALWS   	Mean Scalar Wind Speed (despiked)   [m+1 s-1]
5. SCALWSr   	Mean Scalar Wind Speed (raw)   [m+1 s-1]
6. VECWS    	Mean Vector Wind Speed (despiked)   [m+1 s-1]
7. USPD_rot    	Mean Vector Wind Speed (despiked, rotated/streamwise) [m+1 s-1]
8. VN   	Mean north vector   [m+1 s-1]
9. VE    	Mean east vector   {m+1 s-1]
10. UVAR    	U Variance (despike,detrend,unrotated)   [m+2 s-2]
11. VVAR    	V Variance (despike,detrend,unrotated)   [m+2 s-2]
12. WVAR    	W Variance (despike,detrend,unrotated)   [m+2 s-2]
13. UVAR_rot    U Variance (despike,detrend,rotated)   [m+2 s-2]
14. VVAR_rot    V Variance (despike,detrend,rotated)   [m+2 s-2]
15. WVAR_rot    W Variance (despike,detrend,rotated)   [m+2 s-2]
16. SIGMAT    	Sigma (theta), horizontal   [radians]
17. SIGMAP    	Sigma (phi), vertical   [radians]
18. UV_rot    	u'v' momentum flux (despike,detrend,rotated)   [m+2 s-2]
19. UW_rot    	u'w' momentum flux (despike,detrend,rotated)   [m+2 s-2]
20. VW_rot	v'w' momentum flux (despike,detrend,rotated)   [m+2 s-2]
21. WTBAR_rot   w'T' sensible heat flux (despike,detrend,rotated)   [m+1 K+1 s-1]
22. USTR_rot    u* (despike,detrend,rotated)   [m+1 s-1]
23. OLEN_rot    Obukhov Length (despike,detrend,rotated)   [m-1]
24. TAVG    	Mean Virtual Sonic Temperature (despiked)   [C]
25. TSDEV    	Standard deviation temperature (despike,detrend)   [C]
26. UAVGr   	 Mean U Component Wind Speed (raw)   [m+1 s-1]
27. VAVGr    	Mean V Component Wind Speed (raw)   [m+1 s-1]
28. WAVGr    	Mean W Component Wind Speed (raw)   [m+1 s-1]
29. USDEVr    	Standard Deviation U (raw)   [m+1 s-1]
30. VSDEVr	Standard Deviation V (raw)   [m+1 s-1]
31. WSDEVr	Standard Deviation W (raw)   [m+1 s-1]
32. USTR    	u* (despike,detrend,unrotated)   [m+1 s-1]
33. UV    	u'v' momentum flux (despike,detrend,unrotated)   [m+2 s-2]
34. UW    	u'w' momentum flux (despike,detrend,unrotated)   [m+2 s-2]
35. VW    	v'w' momentum flux (despike,detrend,unrotated)   [m+2 s-2]
36. WTBAR    	w'T' sensible heat flux (despike,detrend)   [m+1 K+1 s-1]
37. UT    	u'T' advective heat flux (despike,detrend)   [m+1 K+1 s-1]
38. OLEN    	Obukhov Length (despike,detrend)   [m-1]
39. UAVG    	Mean U Component Wind Speed (despiked)   [m+1 s-1]
40. VAVG    	Mean V Component Wind Speed (despiked)   [m+1 s-1]
41. WAVG    	Mean W Component Wind Speed (despiked)   [m+1 s-1]
42. TAVGr    	Mean Virtual Sonic Temperature (raw)   [C]
43. TSDEVr   	Standard deviation temperature (raw)   [C]
44. skwU    	Skewness U
45. skwV    	Skewness V
46. skwW    	Skewness W
47. skwT    	Skewness T
48. kurU    	Kurtosis U
49. kurV    	Kurtosis V
50. kurW    	Kurtosis W
51. kurT    	Kurtosis T

In the description below, a cycle refers to a single pass through a single 
record for the specified variable during the despiking process. The column 
headers for the quality control parameter file are:

1.    XXMMDDYYHRMN, XX is the identity of the sonic anemometer, MM is 
              the month, DD is the day, YY is the year, and HRMN is the starting hour
              and minute of the 10 or 30 minute averaging period for that row
2.    Number of observations in the averaging period
3.    Flag=1 if number of observations is more than 100 outside of nominal 10 Hz value 
                  for the averaging period (10-minutes, 5900-6100; 30-minutes, 17900-18100)
4.    Flag=1 if number of spikes in u is greater than 0.5% of observations for any single 
                   cycle
5.    Flag=1 if number of spikes in v is greater than 0.5% of observations for any single 
                   cycle
6.    Flag=1 if number of spikes in w is greater than 0.5% of observations for any single 
                   cycle
7.    Flag=1 if number of spikes in T is greater than 0.5% of observations for any single 
                   cycle
8.    Total (cumulative) number of spikes detected in u after lpknt_U cycles through 
                   record
9.    Total (cumulative) number of spikes detected in v after lpknt_V cycles through 
                    record
10.  Total (cumulative) number of spikes detected in w after lpknt_W cycles through 
                   record
11.  Total (cumulative) number of spikes detected in T after lpknt_T cycles through 
                    record
12.  lpknt_U, 	number of cycles through u record to eliminate all spikes. The 
                   maximum number of cycles allowed is 11.
13.  lpknt_V, 	number of cycles through v record to eliminate all spikes.
14.  lpknt_W, 	number of cycles through w record to eliminate all spikes.
15.  lpknt_T, 	number of cycles through T record to eliminate all spikes.
16.  flgRES_U, 	number of times >70% of bins in 1000 point moving window 
                   amplitude resolution test are empty for u
17.  flgRES_V, 	number of times >70% of bins in 1000 point moving window 
                   amplitude resolution test are empty for v
18.  flgRES_W, 	number of times >70% of bins in 1000 point moving window 
                   amplitude resolution test are empty for w
19.  flgRES_T, 	number of times >70% of bins in 1000 point moving window 
                   amplitude resolution test re empty for T
20.  flgDRP_U, 	number of times >15% of points in u record fall in same bin for 1000 
                   point moving window
21.  flgDRP_V, 	number of times >15% of points in v record fall in same bin for 1000 
                   point moving window
22.  flgDRP_W, 	number of times >15% of points in w record fall in same bin for 1000 
                   point moving window
23.  flgDRP_T, 	number of times >15% of points in T record fall in same bin for 1000 
                   point moving \ window
24.  flgABS_U, 	number of points in u record > 30 m s-1 (check after despiking)
25.  flgABS_V, 	number of points in v record > 30 m s-1 (check after despiking)
26.  flgABS_W, 	number of points in w record > |5 m s-1| (check after despiking)
27.  flgABS_T, 	number of points in T record, T > 45C or T<-30C (check after 
                   despiking)
28.  flgHT1_U, 	number of soft Haar transform threshold exceedances for mean u (2x 
                    threshold)
29.  flgHT1_V, 	number of soft Haar transform threshold exceedances for mean v (2x threshold)
30.  flgHT1_W, 	number of soft Haar transform threshold exceedances for mean w (2x threshold)
31.  flgHT1_T, 	number of soft Haar transform threshold exceedances for mean T (2x threshold)
32.  flgHT2_U, 	number of soft Haar transform threshold exceedances for standard 
                   deviation u (2x threshold)
33.  flgHT2_V, 	number of soft Haar transform threshold exceedances for standard 
                   deviation v (2x threshold)
34.  flgHT2_W, 	number of soft Haar transform threshold exceedances for standard 
                   deviation w (2x threshold)
35.  flgHT2_T, 	number of soft Haar transform threshold exceedances for standard 
                   deviation T (2x threshold)
36.  flgHT3_U, 	number of hard Haar transform threshold exceedances for mean u (3x threshold)
37.  flgHT3_V, 	number of hard Haar transform threshold exceedances for mean v (3x threshold)
38.  flgHT3_W, 	number of hard Haar transform threshold exceedances for mean w (3x threshold)
39.  flgHT3_T, 	number of hard Haar transform threshold exceedances for mean T (3x threshold)
40.  flgHT4_U, 	number of hard Haar transform threshold exceedances for standard 
                   deviation u (3x threshold)
41.  flgHT4_V, 	number of hard Haar transform threshold exceedances for standard 
                   deviation v (3x threshold)
42.  flgHT4_W, 	number of hard Haar transform threshold exceedances for standard 
                   deviation w (3x threshold)
43.  flgHT4_T, 	number of hard Haar transform threshold exceedances for standard 
                   deviation T (3x threshold)
44.  flgSKW_U, 	flag=1 for |u skewness| > 1; flag=2 for |u skewness| > 2
45.  flgSKW_V, 	flag=1 for |v skewness| > 1; flag=2 for |v skewness| > 2
46.  flgSKW_W, 	flag=1 for |w skewness| > 1; flag=2 for |w skewness| > 2
47.  flgSKW_T, 	flag=1 for |T skewness| > 1; flag=2 for |T skewness| > 2
48.  flgKUR_U, 	flag=1 for u kurtosis < -1 or u kurtosis > 2; flag=2 for u kurtosis < -2 or 
                   u kurtosis > 5
49.  flgKUR_V, 	flag=1 for v kurtosis < -1 or v kurtosis > 2; flag=2 for v kurtosis < -2 or 
                   v kurtosis > 5
50.  flgKUR_W, 	flag=1 for w kurtosis < -1 or w kurtosis > 2; flag=2 for w kurtosis < -2 
                   or w kurtosis > 5
51.  flgKUR_T, 	flag=1 for T kurtosis < -1 or T kurtosis > 2; flag=2 for T kurtosis < -2 or 
                   T kurtosis > 5
52.  flgRNU, 	RN alongwind relative nonstationarity test for u; flag=1 for RNU > 0.5
53.  flgRNV, 	RN crosswind relative nonstationarity test for v; flag=1 for RNV > 0.5
54.  flgRNS, 	RN vector wind relative nonstationarity test wind speed; flag=1 RNS>0.5
55.  flgRSET, 	flag=1 for relative systematic flux sampling error test (RSE) > 0.5
56.  RSET, 	value for RSE
57.  flgRFET, 	flag=1 for random flux sampling error (RFET) test value > 0.25
58.  flgRNT, 	flag=1 for flux trends associated mesoscale motions (RNT) value > 0.25
59.  RFET, 	value for RFET 
60.  RNT, 	value for RNT

Finally, there is a group of text files containing the raw, unprocessed 10 Hz 
data for the ARLFRD sonic anemometer measurements. They are reported in a series 
of data records of 4 hours each. The data covers one hour before the IOP tracer 
measurement period to to one hour after the tracer measurements. There is one
file per sonic anemomemter per IOP. The filenames are 'PSB2_SS_htm_IOPn.txt'
where 'SS' is the sonic designation (e.g., R1, G2), 'htm' is the agl height of
the sonic in meters, and 'n' is the number of the IOP. There is no file for the 
G2 sonic for IOP4. 

The R.M. Young sonic files (SS=R#) have the following columns:

1.    date (mm/dd/yy)
2.    time (hh:mm:ss)
3.    U wind component (m s-1)
4.    V wind component (m s-1)
5.    W wind component (m s-1)
6.    T (deg C)

For sonics R1 and R9 the date has a 4-digit instead of 2-digit year. The G2 sonic 
(SS=G2) has the same date and time columns as the R.M. Young sonics. The next three 
numeric fields are U, V, and W (m s-1). The next column is the speed of sound in air 
(C_air). For sonic G2 in IOPs 1 and 2, the times reported for the raw, unprocessed
data are MDT but the data represents the same time period as the other sonics. This 
is the only exception to the use of MST in all other data sets.

Reference

Vickers, D., and Mahrt, L., (1997), Quality control and flux sampling problems 
for tower and aircraft data. J. Atmos. Oceanic Technol., v. 14, p. 512-526.
