A December 2015 audit report by the Energy Department’s Inspector General’s Office deserves a little auditing itself.
The audit examines the issues management program at the Los Alamos Field Office (LAFO), which falls under the National Nuclear Security Administration, a semi-autonomous agency within the Department of Energy. NNSA is mainly responsible for the national security activities of DOE, including the nuclear weapons portfolio, the nuclear stockpile, threat reduction, non-proliferation and a variety of science programs that contribute to those missions.
The audit finds a number of deficiencies in the LAFO program, mainly record-keeping details and incomplete tracking information, having to do with three basic categories – a corrective action program to identify and rectify deficiencies, an employee concerns program (ECP) that “allows employees to raise concerns and have those concerns independently investigated without fear of retaliation,” and a process to resolve differing professional opinions (DPO)
The audit took place over a two year period from January 2014 to December 2015 and covered issues management records from 2009 through 2015. It is unclear how many auditors there were, because the pronoun “we” is used throughout, and “we” too can imagine that more than one person has been involved. In any case, the auditors managed to select 102 of 310 records for review during the two year audit and the conclusions are based on analysis of less than a third of a rather small number of records.
Oddly enough, the audit found only three instances of problems in the professional opinions category.
“Nothing came to our attention to indicate that the process was not generally effective in managing the DPO,” the audit states, using a double negative, which generally connotes an equivocal positive. While the auditors checked into the personnel records of individuals involved in the three cases to obtain the conclusion forms that were missing from the issues management records, the auditors suggest an alternative might have been more effective if there were a DPO log where tracking numbers were assigned, so that a missing file could be found wherever it ended up
Why were there so few professional disagreements? Hard to say, but one reason touched upon was that the Field Office didn’t keep these issues in the issues management data base, but handled them separately as personnel matters, which raises the question of a feedback loop that might inhibit differences of opinion. The audit also expresses a concern that that there is no record of rejected DPO cases, so “there is no assurance that the Field Office has a complete record of submittals.”
Here’s the problem with this, which may be the problem with audits in general. There is very little qualitative information to go by, not enough stories with people in them. We learn instead, for example, that 22 of 27 deficiencies (81 percent) that originated from prescribed Field Office self-assessments were entered more than 90 days after the assessment report date.
Occasionally, almost by accident we come up actual anecdote, that a certain facility has requested an exemption from a fire safety provision rather than correcting the problem, and then said that it was corrected when it wasn’t. Not that facilities do such things, rather than the people who work there.
Finally, we don’t learn very much about what is really going on. Procedural errors, unfinished documentation, inaccurate data, yes, but these are just numbers, percentages, stripped of their real content. True, it’s a starting point for benchmarking, or establishing a baseline for trending performance, but it’s a frail reed for the momentous task at hand.
Even more to the point, the auditor makes no bones about the magnified consequences that may result from seemingly harmless deficiencies when the paperwork or database entry that reflects the human dynamics at work originates in nuclear facilities. But without more qualitative balance for the quantitative indicators, it’s hard to know when the juice is worth the squeeze.