Ever since the Eisenhower administration, the American Bar Association has enjoyed a preferred seat at the judicial nominations table. For decades, presidents allowed the ABA to vet nominees even before they were announced. Today, the ABA continues to evaluate and interview nominees after their nominations and publically submit its rating to the president and the Senate Judiciary Committee.

At one time, the ABA’s ratings system was considered the “gold standard” for rating judicial nominees. But the ABA’s rating system has fallen on hard times. Although conservatives have historically criticized the ABA for its alleged left-leaning bias, recent evaluations indicate that critics may have a point.

Consider the ABA’s recent rating of Ninth Circuit nominee Lawrence VanDyke. Submitted by letter the night before his Senate Judiciary hearing, the ABA determined that VanDyke was “Not Qualified” to be a federal appellate judge. The cursory two-page letter acknowledged his professional accomplishments as an attorney with nearly 14 years’ experience—including stints as solicitor general of two states—but cited anonymous interviewees who claimed he was “arrogant, lazy, an ideologue, and lacking in knowledge of the day-to-day practice including procedural rules.” The letter also alleged there were questions that VanDyke would be fair to advocates before him, including members of the LGBTQ community.

“Not qualified” ratings by the ABA are rare, but nevertheless serve as a strong, public rebuke against a judicial nominee by a 400,000 member organization. Such ratings are also embarrassing and too often gleefully used by critics to mock a nominee they otherwise know little about. Given the ABA’s power and unique influence, when the ABA labels a nominee “Not Qualified,” it has a responsibility to explain why and how it reached that decision. On these prongs, the ABA has failed spectacularly.

First, in evaluating VanDyke, it is unclear whether the ABA even followed its own policies. For instance, the ABA’s letter was sent the evening before VanDyke’s morning hearing, well after the ABA’s own self-imposed 24-hour deadline. Further, when an ABA evaluator considers a “Not Qualified” rating, ABA policy requires a second evaluator to review the information collected by the first evaluator and again interview the nominee. It seems neither step was taken here.

Next, given the letter’s substantial indictment against VanDyke’s character and professional abilities, it contains a disappointing lack of transparency. The letter gives no indication of how many interviewees actually raised concerns about VanDyke’s purported arrogance and ignorance or how many offered universal praise. Complicating matters further, several interviewees have since publically stated that they raised no concerns over VanDyke’s nomination during their interviews. For his part, during his committee hearing, VanDyke tearfully denied the letter’s characterization of his views. Yet the ABA has refused to release a transcript or notes of his interview with the ABA evaluator.

These defects are alarming. Given its power and unique influence, the ABA has a responsibility to do better.

The most important step the ABA can take is also the simplest—consistently follow its own policies for judicial evaluations. This means that if the ABA is considering a “Not Qualified” rating for a judicial nominee, a second evaluator must consider the evidence. This allows a more thorough review and prevents any perceived or actual conflicts of interest between the evaluators and the nominee. It also means that—after multiple reviews—if the ABA believes a nominee is not qualified, it must timely submit its rating. No more last-minute letters.

The next step is similarly straightforward: The ABA should increase the transparency of its “Not Qualified” evaluations. Today, ABA evaluators spend hundreds of hours interviewing colleagues and reviewing a nominee’s past work. This information is reviewed only internally, and the association’s final “Not Qualified” ratings are only summarized in a short public letter.

Given the ABA’s privileged standing and considerable public voice on judicial nominations, it must do more. For negative evaluations, it must explain specifically what public works the evaluators found problematic, approximate the number of interviewees that raised concerns about a nominee’s abilities, and provide more context on the interviews that occur between nominees and ABA interviewers. Doing so is possible without compromising interviewees’ anonymity or dissuading them from speaking candidly.

The VanDyke case further strains the ABA’s credibility as an impartial evaluator. But the ABA can take steps to fix this. It can strengthen the transparency of its evaluations or, perhaps more preferably, it can get out of the judicial evaluation game altogether. If not, the Senate has no obligation to keep its seat warm at the nominations table.