Interview transcript:
Stephen Bacon We’re seeing a lot more protests, particularly at the Government Accountability Office, in the last several months that have been filed using AI — a party that’s not represented by counsel using AI to generate the protest and then file it. But we’re seeing some problems with that in some of the decisions that are coming out of the GAO.
Terry Gerton So tell me more about how companies are using AI. You mentioned that they’re doing this without the help of legal counsel as well.
Stephen Bacon That’s right, at least the ones that we’ve seen so far in public decisions at GAO. It’s not entirely clear how the protesters are using it, but we can imagine that maybe they’re taking the debriefing information that they’re getting from the agency, they’re uploading that into an LLM like ChatGPT or Claude, and using it to develop a protest argument that they can file with the GAO. And what we’re seeing in the decisions is that many of the protests that have been filed using AI contain hallucinations. Case citations that don’t exist to actual cases that have decided by GAO. So the legal precedent that the protesters are relying on, in fact, don’t exist. And that’s one of the inherent limitations of LLMs is that they hallucinate. They come up with decisions, citations that don’t exist. To be clear, we’re not just seeing this by protesters that are not represented by counsel. This is happening in courts all across the country where attorneys are using AI to help generate legal filings and then getting in trouble with the courts when those citations don’t actually exist. Because when you file a protest or any kind of legal filing that has a citation in it, the court is relying on you to make an accurate representation that the legal authority that you’re relying on is in fact correct and is in fact a decision that has been issued in the past. And so both courts and the GAO now are saying that you can get in trouble, you can be sanctioned as a protester if you submit a protest that has some kind of fake citation that’s inaccurate.
Terry Gerton What does that mean to be sanctioned as GAO reviews the case?
Stephen Bacon At GAO, they have inherent authority to sanction protesters, and really the main sanction that they have is to dismiss a protest. If you happen to file a protest that contains fake citations, they reserve the right to dismiss your protest. Even if you have legally valid grounds to protest — maybe you have identified an error in the agency selection process — if GAO determines that you relied on fake citations in your protest, they could dismiss the protest, even if the actual merits of it may have some validity to it.
Terry Gerton So there’s some interesting intersections of situation going on here, I think. There’s a lot of uncertainty on the contractor side about the new FAR regulations and how those are going to be enforced, certainly across different agencies. We’ve had a reduction in the contractor workforce, so there are fewer contractors managing more acquisitions. And now we have AI coming in to sort of simplify, but potentially also make much more complex, the whole protest market. So do you expect all of this to be leading to an increase in protests? And what does that mean for GAO as they’re trying to sort out the validity of all the claims?
Stephen Bacon I think it certainly has the potential to, if what we’re seeing in the decisions is a trend towards more pro se protesters — pro se being parties that are not represented by counsel — using AI. To the extent that that trend continues, I think that there’ll be a lower barrier for protesters to file at GAO if they think that they can use an LLM to generate a protest without having to spend legal fees on outside counsel. Which is understandable, particularly for small businesses who may have resource constraints. If they feel like they can use an LLM to help them challenge an award decision, we may see more of that at GAO. I think what GAO is saying in these opinions that have come out…at first, they’re warning protesters that using LLMs that create fake citations is sanctionable. They didn’t actually take the step of issuing a sanction. But finally, in the last several months, we saw that they did, in fact, take that step of dismissing a protest, actually several protests that were filed by the same company, that contained fake citations. They actually took that step and dismissed those protests on the grounds that they misrepresented legal authority in their filings with the GAO.
Terry Gerton I’m speaking with Stephen Bacon. He’s a partner in the government contracts practice group at Rogers Joseph O’Donnell. You mentioned small businesses and their capacity constraints in terms of they may not have in-house counsel, they may have a lot of folks who can review all of this. But does this have the effect of sort of adding some equality into the protest market where they can use AI to submit? And do you think then that that’s going to change the protest space? Is this just the tip of the iceberg in terms of transformation?
Stephen Bacon It certainly lowers the barrier for companies. The GAO was set up to be a relatively informal forum to allow for the quick and efficient resolution of protests. I don’t think what GAO is saying necessarily is that AI cannot be used. But what they are saying is that we have a process to resolve bid protests and we want to maintain the integrity of that. And if you’re going to use AI, you need to be sure that you verify that what you’re filing is accurate. For anybody that is thinking about using AI to generate a protest, there needs to be some level of quality-checking of what is in the draft that’s generated by an LLM to be sure that you’re making accurate representations to the GAO in your protest. So that means checking the legal citations to make sure that the cases actually exist. That basic level of quality-checking needs to happen. Otherwise, GAO could just be flooded with protests that have no merit and that have lots of inaccuracies in them. And that’s not going to help them resolve protests in a way that’s efficient and achieves their ultimate goal.
Terry Gerton So where do you think we go from here, and what’s your guidance to the companies who are considering using AI to file their protests?
Stephen Bacon For any company that’s contemplating using an AI to generate protests, the basic point: If you’re going to do it, you have to verify that the citations are accurate. You have verify that what an LLM is generating is citing to a decision that has been published by GAO in the past. And that’s relatively easy to do. GAO has all of their decisions on their website, and you can go and check those and verify not only that the citations are accurate, but the legal proposition that you’re asserting is supported by the case that’s being cited. That’s important, too. That’s kind of table stakes. But the other thing I would say is that what we’re seeing in a lot of these decisions, where it’s obvious from the decision that AI has been used and that GAO is pointing out that there are these fake citations, is that oftentimes those protests are being dismissed for procedural defects as well. So things like timeliness and bid protest standing. Those kinds of procedural issues are being missed by the protesters who are using LLMs to generate the filings. And that’s because of another inherent limitation of an LLM; it often will tell you what you want it to say in a lot of ways. So if you tell the LLM, generate me a protest on this issue or that issue, it will do that and it might produce something that looks, on its face, credible and compelling. But if you don’t have the domain knowledge of the timeliness rules and the standing rules, you’re often going to overlook those things and the LLM is not going to catch it for you. And so you may be in a situation where you file something that looks on its face credible, but is in fact an untimely protest.
The post AI-crafted bid protests are on the rise, but what’s the legal fallout? first appeared on Federal News Network.