Forensic commission will investigate Austin PD drylabbing allegations

Yesterday Grits offered up an account of the investigative panel on the El Paso crime lab at the Forensic Science Commission, and I chose that subject in part because Chuck Lindell at the Austin Statesman was there to cover the other big story of the day: The Commission's decision to open an investigation in response to allegations surrounding the Austin crime lab. (There appeared to be no MSM reporters, even from the El Paso Times, at the 7 a.m. investigative panel.) Reported Lindell ("State panel opens inquiry into Austin police crime lab," April 14):

The Texas Forensic Science Commission voted unanimously Friday to open an investigation into two complaints about the Austin Police Department crime lab's testing of drug evidence.

A three-member subcommittee will lead the investigation — holding meetings that will be open to the public — and report its findings to the full commission, which will conclude the case with a written determination.

The state commission, which investigates allegations of negligence or misconduct involving forensic labs, set no deadlines during Friday's meeting in Austin.

One complaint against the Austin police crime lab was filed by another facility, Integrated Forensic Laboratories in Euless, which questioned testing results or procedures used on evidence in three cases in which it followed up on work by the Austin lab. ...

The second complaint was filed by Debra Stephens, a scientist who worked for several years at the Austin crime lab until she was fired last year.

Lindell goes on to give topline accounts of the specific complaints being investigated, but the most interesting to me were allegations of "drylabbing" preliminary reports. The Austin PD crime lab told the Commission that retesting in one case out of the 23 identified by Ms. Stephens found a substance originally reported that was not there on retesting, but said it didn't affect the outcome of the case. The Commission will have to get to the bottom of that dispute. Stephens told Lindell otuside the meeting that the error "led to a charge being dropped." "I'd call that impacting the outcome of the case," she told him, alluding to discussions over the Commission's narrow definitions of negligence and misconduct.

One tidbit Chuck mentions only in passing deserves more explication. It came out in the discussion that, during the period under review, Austin crime lab workers would simply record results on scraps of paper - often sticky notes - then throw the only original documentation away after performing a test, merely entering the results into the computer system. In one instance, documentation provided by Ms. Stephens showed lab tests continued to be run six minutes after a preliminary report was issued to law enforcement. According to DPS, the practice did not meet ASCLD/LAB standards (they cited the specific regs, but I didn't jot them down), and APD said it changed protocols to eliminate the practice of issuing preliminary reports. They now require testing to be completed before reports are issued, though you'd kinda think that should have been obvious in the first place.

Another interesting allegation against APD came from a private crime lab in Tarrant County (the same one whose director is serving half-time overseeing the El Paso lab) involving whether to classify a substance as "marijuana" or as "THC" found in a substance "other than" marijuana, which receives a higher penalty. A rep from the Austin Bexar County crime lab was unhappy this allegation had been brought, declaring emphatically that the FSC wasn't the right place for resolving disputes between lab interpretations. But with so many other allegations bundled up in the same batch, his pleas not to examine the lab's practices on that score fell on deaf ears. It's possible, commissioners said, it could be a legal instead of a scientific dispute, but they couldn't make that judgment without digging into it further.

Obviously, Grits will continue to track this subject as the FSC inquiry moves along.

Ghost of John Bradley's chairmanship haunts El Paso crime lab inquiry

The ghost of John Bradley's Forensic Science Commission chairmanship rose from the grave to haunt another FSC investigation this morning, as the Commission relied on a narrow definition of "professional negligence," suggested and installed in agency policy by Bradley, to say the El Paso crime lab wasn't guilty of it.

The state law creating the forensic commission did not define negligence and did not give the FSC rulemaking power. But regular readers may recall Mr. Bradley pushed the commission at his first meeting to create supposedly nonbinding policies governing their activities, which he proposed whole hog without input from other commissioners. Under the definition of "professional negligence" which Mr. Bradley proposed and the commission regrettably adopted unedited (despite concerns raised by their Attorney General adviser that it was too narrow), the FSC may not issue a finding of negligence unless "the negligent act or omission substantially affected the integrity of the results of a forensic analysis" (emphasis added). According to new Chairman Vincent Di Maio, and the rest of the commission eventually agreed, that means that unless the results of a lab analysis were incorrect, they could not find negligence. In the end, the Commission interpreted the definition to say they could not hold the El Paso crime lab accountable for "negligence" because none of the samples processed by an incompetent analyst came up incorrect when retested - sort of the forensic version of "harmless error" in appellate court settings.

It's certainly comforting that no errors that we know of resulted in false convictions, but it's still negligent to employ incompetent staff, still negligent to disempower the Quality Assurance manager from improving practices. According to the accrediting body ASCLD/LAB (see their 2011 report [pdf]), the agency's division responsible for controlled substances testing employed incompetent analysts with inadequate supervision, overstated the thoroughness of testing, failed to follow their own laboratory policies, failed to base laboratory procedures on known standards, failed to document testing/calibration of instruments, allowed improper access to the lab by police officers and other personnel, and failed to require a written test to determine labworker competency. At one point, two different people independently believed they were the Quality Assurance manager (it's still confusing who was right). As a FSC commissioner said today, the lab was an "accident waiting to happen."

Now, to be fair, by all accounts the El Paso crime lab, particularly the division in question, has undergone massive change for the better. Two of the three analysts in the division are gone - including one deemed incompetent through proficiency testing - and the lab hired a private consultant from Tarrant County to serve as acting lab director on a half-time basis. At this point, said a DPS auditor, the lab is doing quality work in compliance with accreditation standards. El Paso PD and the District Attorney, who was present, were praised for their reaction to the situation, including their cooperation with regulators and the DA's notification of defense counsel in affected cases. EPPD told the commission they were trying to find a permanent lab director but may also yet decide to outsource controlled substance testing entirely. DA Jaime Esparza told the Commission he appreciated their hard work and believed their thoroughness would increase "community confidence."

So whether or not the FSC found negligence, the process clearly improved things for the better. ASCLD/LAB had taken the lab off probation in December, but a DPS audit commissioned by the FSC found more problems to address, with DPS suggesting "significant improvements" in February which have all since been adopted, the commission was told. The lab has undergone a "massive transition" since DPS visited, the auditor said, which made one wonder if their probation perhaps ended prematurely if there was that much more to be done.

What is negligence?
Regardless, the FSC is clearly using a flawed definition of negligence and misconduct. Commissioners should change it. As Dr. Sarah Kerrigan pointed out, under the interpretation propounded today one couldn't find misconduct in a "drylabbing" case (submitting reports without doing the testing). For example, a new, self-reported FSC case out of Tarrant County involves a crime lab employee who failed to test rape kits when the police report said no penetration occurred, claiming to have performed tests in at least five cases when he did not. When the kits were tested, indeed, they contained no semen, so the "results" weren't any different. But is that not still misconduct by the lab worker?

Di Maio insisted such a case of overt fraud was clearly "misconduct," but the FSC policy on misconduct policy contains the same language as the "negligence" definition about "results," so if they can't find El Paso negligent without an incorrect "result," it's hard to understand how a drylabbing case would be any different except in wrongful conviction cases where someone was falsely accused.

Even Di Maio, however conceded the El Paso lab as recently as last year faced serious problems before the recent shakeup, stemming he said mostly from administrative failures, failure to follow Quality Assurance guidance (again, not negligence?), and a lack of "scientific leadership." "Scientific problems, yeah,"said Di Maio, "but those were because of the management problems."

Praising ASCLD/LAB
I was especially interested and rather curious to hear Commissioner Sarah Kerrigan go out of her way to exonerate accrediting agency ASCLD/LAB in the El Paso incident, declaring that in this case the accreditation process was "working and functional." This struck me as an extraordinarily odd statement, and perhaps a bit of a forced one. Yes, it was ASCLD/LAB's report last year that brought these allegations to a head, and their investigation was much more thorough than, say, the one described in this Grits post, in part because the El Paso lab was transitioning off of its "legacy" accreditation to new international "ISO" standards just this month, requiring a much more complete vetting of the lab. But by all accounts, most of the same problems were raised by the accrediting body in 2006, and for five years ASCLD/LAB accreditation did nothing to fix them. Better late than never, I suppose, but for those five years that accreditation wasn't a particularly meaningful moniker. At best it's a mixed result for ASCLD/LAB.

By contrast, the Forensic Science Commission's vetting process, particularly when coupled with DPS serving as their audit arm, both investigated the case thoroughly - even if their judgment in the end was based on a skewed standard - and the process appeared to get the attention of city and county leaders in a way that the accrediting body could not in the five years after the problems were first identified in 2006. I expect this report, which will be vetted in full at the next FSC meeting, in that respect may end up a bit like the Todd Willingham arson report, making no misconduct or negligence finding but then describing at some length activities that seem, upon description, at a minimum negligent and which required extensive remediation. This process is opening up a unique and fascinating window onto crime lab problems and solutions, launching discussions that otherwise wouldn't or couldn't occur in a public setting.

Indeed, despite my complaint about John-Bradley-spawned problems with the FSC's definitions of negligence and misconduct, in general Texas' Forensic Science Commission appears to be finally hitting stride. Most commissioners seem to want to do a good a job, which is half the battle, and the staff have shown their competence at undertaking rather large-scale evaluations with impressive diligence and a fair but thorough approach. At times during the Willingham saga, many of us who work toward forensic reform have wondered if the agency would ever find its purpose and function, but they appear to have largely done so, even if the specter of John Bradley's chairmanship still looms over their work. The truth is, even if the FSC finds negligence or misconduct, there are no "teeth" to their authority. They cannot punish crime labs. They can only expose negligence, misconduct, and junk science to the light of day. And as long as they do that effectively, maybe for now it doesn't matter so much what they call it.

MORE: (4/14) Rereading this post, Grits should elaborate on the comment that the Attorney General legal adviser to the FSC worried the definition of "negligence" was too narrow at the time it was created. Grits was liveblogging that day's meeting back in January 2010 and wrote at the time:

(12:05): The AG rep described a "small gap" in the negligence and misconduct definitions (I don't have a copy so I can't be specific) that includes people who are aware of professional standards and fail to follow them but where the incident may not rise to the definition of "negligence" in the rules - a category the AG's rep said would likely make up a large number of cases submitted to the commission. The scientists on the committee didn't seem to understand the significance of what was being said to them on this score (and the defense attorney's rep isn't there), so they moved forward without changing the language or closing the "gap." Ouch!

Those definitions have remained the same in FSC policies as they were when Mr. Bradley forwarded them to commissioners the day before that meeting in Harlingen a little more than two years ago. Complaints arose at that 2010 meeting that the most controversial sections of Mr. Bradley's rules, including these definitions, were also the portions with no references to sources and did not rely on definitions and standards either from other jurisdictions or the feds. In the end, though, they adopted them anyway, a decision which rose up yesterday morning like some old, tired, poltergeist emanating from Bradley's misbegotten chairmanship, not as ominous or imminently scary as before, perhaps, indeed unseen by all but a few, but present nonetheless, hovering over the proceedings, smiling.

RELATED: Shortcomings in crime lab accrediting body supply role for forensic commission.

Probing probes by ASCLD/LAB: Conference calls substitute for investigation

Grits has discussed recently whether "accreditation" of crime labs provides meaningful oversight, citing an appeal last year to the New York Forensic Science Commission urging them to stop using ASCLD/LAB, which is also the main body that accredits crime labs in Texas. Attorney Paul Kennedy at The Defense Rests provides an example of ASCLD/LAB's oversight in the case of the Southwest Institute of Forensic Sciences in Dallas after allegations that "the lab has been using chemicals after the expiration dates on the bottles, that the lab is using outdated protocols and that analysts have been conducting tests without wearing gloves (among others)."

Their investigation method: They initiated two conference calls with lab management and issued a report parroting their responses as findings without even contacting the complainant. Writes Kennedy:

ASLD took over 13 months to complete their "investigation." Their investigation consisted of telephone interviews with managers at SWIFS (Southwest Institute of Forensic Sciences) in Dallas. Not once did anyone from ASLD contact the individual who made the complaint.

After ASLD's report was issued, it took the Texas Forensic Sciences Commission another 13 months to forward the findings to the individual who made the complaint. As a result of the "investigation" by the body who had accredited the lab two years prior to the complaint, the TFSC found the complaint to be groundless.

Not much of an inquiry if they never spoke to the complainant nor verified verbal responses from the lab. Then the Forensic Science Commission (belatedly) relied on the ASCLD/LAB findings - which really represented barely any investigation at all - to dismiss the complaint. (See the report [pdf].) Not encouraging, huh?

To be fair, the delay under the forensic commission occurred under John Bradley's chairmanship while the group was distracted, divided, and embroiled in the Todd Willingham arson controversy. And despite the poor quality of the Dallas investigation, Grits can't say for certain a more thorough one would have found anything. But this episode demonstrates why mere accreditation - or the say so of ASCLD/LAB - isn't necessarily sufficient to conclude a crime-lab complaint was unfounded. And if that's the case, what good are they?

MORE: The Forensic Science Commission's investigative panel on the El Paso crime lab meets tomorrow morning at 7 a.m. in Austin, for the early birds among you. See the agenda (pdf), as well as the one (pdf) for their regular meeting at 9:30. ASCLD/LAB took El Paso off probation, readers may recall, before all its problems had been addressed.

See related, recent Grits posts:

• Shortcomings at crime lab accrediting body supply role for forensic commission
• Is ASCLD/LAB crime-lab accreditation inadequate?
• The fundamental role of states in crime lab regulation

Shortcomings at crime lab accrediting body supply role for forensic commission

As Texas' Forensic Science Commission evolves in how it does its work in response to the actual, real-world complaints it receives, it's increasingly clear one of its primary roles will be to plug the gaps created by shortcomings in crime labs' accreditation process. Last week, Grits posted a detailed critique (pdf) of ASCLD/LAB, the crime-lab accrediting agency, submitted last year to the Forensic Science Commission in New York by a prominent attorney who criticized the group for lax oversight. He recommended NY state find a new accrediting body, as the United States Army Criminal Investigation Lab recently did following problems at an accredited lab.

At the Texas Forensic Science Commission, the Willis and Willingham cases involved arson, which is not an ASCLD/LAB accredited discipline. But more recent cases demonstrate why the TX FSC and ASCLD/LAB may end up butting heads in some instances, perhaps sooner than later, over the same issues identified in New York.

The best example may lie in the FSC's investigation of the controlled substances division at the El Paso crime lab. At their January meeting, serious unresolved concerns were raised about the lab and the commission initiated a detailed investigation with the help of the Department of Public Safety. I've uploaded a copy of the letter sent to the EPPD crime lab detailing the extensive checks they demanded be run to ensure the problems had been solved. These included hiring a qualified lab director, retesting all lab work performed since November 2011, and retesting ALL the work ever performed by an analyst found to be incompetent to perform the job, "giving priority to the 60 cases on the DPS list with the greatest possible impact."

The El Paso crime lab was accredited under "legacy" provisions by ASCLD/LAB, meaning they didn't have to meet every last requirement for their initial accreditation. However in April their "legacy" status runs out and the El Paso lab must meet much more rigorous ISO standards or risk losing their accreditation altogether. Ironically, in that light, it came out at the FSC's January meeting that ASCLD/LAB had lifted the probationary status of El Paso's controlled substances division, despite all the looming questions remaining about their competence, lack of leadership, and apparent ill-preparedness to meet the new standards required of them next month. As Grits reported in January:

while most of the media attention has focused on a single lab worker who couldn't pass basic competency tests, Commissioner Sarah Kerrigan, to the nodding affirmation of her peers, strongly urged that that lab worker not be used as a "scapegoat" to avoid bigger changes. When the El Paso lab began its certification process in 2006, the accrediting body found a list of shortcomings they asked them to fix, and which the lab claimed to have resolved. In 2011, though, when the incompetent lab worker came to light, a new assessment identified virtually all the same problems at the lab, still unresolved, that were cited in 2006.

A representative of ASCLD/LAB told the commission that lifting probation didn't mean all the lab's shortcomings have been rectified, which left everyone in the room wondering exactly what it did mean. Looking at the FSC letter detailing the oversight which still hadn't been performed when the lab's probation was lifted, one wonders what regulatory benefit, precisely, that ASCLD/LAB is providing and whether it really has any teeth.

Texas is one of only a few states that requires accreditation of its crime labs, and certainly having some standards to follow is better than flying blind. But standards unenforced are mere suggestions, and it's unclear what besides suggesting the accrediting body does when the labs it regulates don't follow its rules. If the FSC becomes the primary entity regularly performing meaningful crime-lab oversight, it will create a lasting, important role for the commission, particularly over the next decade or so as Texas and the rest of country struggle to correct deficiencies in what has historically passed for forensic science in the courtroom.

The Texas Forensic Science Commission - which at times during the Willlingham/Willis saga seemed to struggle to define its mission - appears to be hitting its stride now that the Culture War circus has left town, they have a chair who actually wants the agency to succeed, and they've found a role to play that the crime labs' accrediting body seems unable or unwilling to embrace.

Is ASCLD/LAB crime-lab accreditation inadequate?

A reader forwarded me this remarkable 31-page memo (pdf) submitted last year to the New York Forensic Science Commission severely criticizing ASCLD/LAB, which is the primary accreditation body for American crime labs, including in Texas. The memo's' author was Marvin E. Schechter, who several months later was named chair of the criminal justice section of the New York State Bar. I've always thought ASCLD/LAB's approach seemed a bit squishy and lab-friendly, but Scheichter's memo questions whether lameness too often extends to complicity, and even whether it's appropriate to rely on the private accreditation body at all:

The repeated instances of nationwide lab failures at facilities under ASCLD/LAB accreditation combined with the severity, scope and magnitude of the North Carolina SBI Laboratory scandal, the pending legislative reforms in North Carolina and the San Francisco DNA mix-up/cover-up warrant that the CFS examine precisely what role ASCLD/LAB plays in forensic review, its methodology, the design of its model and the very integrity of the organization itself, including but not limited to potential, if not actual conflicts of interest. Further there must be a serious discussion of whether the CFS can continue to rely on ASCLD/LAB as an accrediting agency.

I certainly hadn't realized until reading this piece that ASCLD/LAB is no longer  the accreditation body for the United States Army Criminal Investigation Laboratory (USACIL), which dropped ASCLD/LAB after it was reaccredited despite one of its examiners having "engaged in repeated misconduct known to supervisors." ASCLD/LAB only required an internal audit in response and did not publicly acknowledge the problem until it was outed in a newspaper series years later.

Equally damning, as we debate prosecutorial misconduct and Brady issues here in Texas, is the accreditation body's lax attitude toward notification of defendants or sometimes even prosecutors when crime lab errors are discovered:

Transparency does not include notification to District Attorneys (San Francisco, Nassau County) when laboratories engage in misconduct. It does not mean notification to defense attorneys in cases where the representation of their clients is affected. It would appear to be ASCLD/LAB’s position that notifying anyone other than an affected laboratory is not how transparency, or for that matter accreditation, should be viewed.

Prosecutors can't hand over Brady material they never see, just as defendants can't challenge flawed forensic evidence if its imperfections are concealed.

I've heard bits and pieces of these critiques in isolation over the years, but it's stunning to see them all together marshaled into a coherent argument for the first time (for me, anyway). Given the weight afforded to private accreditation in Texas (and other states which have legislated crime lab reforms in the 21st century), anyone with more than a passing interest in crime labs should read the whole thing (pdf). The implications if ASCLD/LAB accreditation really is fundamentally inadequate are significant indeed.