Three-Phase Debriefing Approach: What is Not Measured Cannot Improve
Losing a federal procurement opportunity can be a difficult time for government contractors. Nonetheless, if the procurement effort is conducted under FAR Part 15 or for qualifying task and delivery orders, the agency is required to provide contractors with a debriefing. [1] Debriefings provide unsuccessful offerors and awardees with specific minimal information pertaining to the procurement that is not otherwise prohibited. [2] At a basic level, debriefings may assist contractors in understanding what aspects of their proposal the agency favored or disfavored. That singular aspect of the debriefing process may shed some light on what a contractor can do to improve performance in the future. Additionally, debriefings also enable contractors to assess whether the procurement was conducted in accordance with the terms of the solicitation and within the bounds of applicable law.
While contractors can use debriefings to improve their likelihood of future success or to formulate bid protest strategies, agency officials often learn from the debriefing process as well. Mistakes uncovered during the debriefing process by diligent contractors help procuring agencies reduce the amount of costly litigation and the loss of efficiency ultimately caused by those mistakes. In essence, debriefings provide a mechanism for both contractors and agencies to improve upon the oftentimes complex and convoluted process of public procurement. For example, some agencies have implemented enhanced debriefings, which go beyond the minimum requirements outlined in FAR Part 15. [3] During enhanced debriefings, agency officials typically provide contractors with a variety of documents used in the source selection decision, including all or portions of the evaluation record, agency memoranda, or other information used during the selection process. As a result of offering a more transparent debriefing, at least one agency has seen a reduction in the number of speculative bid protests, which often lead to costly litigation and unnecessary delays. [4]
To make the most of your debriefing, contractors should approach the debriefing process in three phases: (1) pre-debriefing strategy session, (2) the debriefing, and (3) post-debriefing analysis. Throughout the debriefing process, it is critical for contractors to consider the following maxim: what is not measured cannot improve. [5] These steps can help ensure contractors will get the most out of the debriefing process.
Pre-Debriefing Strategy Session
REVIEW THE SOLICITATION AND YOUR PROPOSAL
Before the debriefing, contractors should read, re-read, and read again both their submitted proposal and the agency’s solicitation. Critically assess precisely what was evaluated (and how it was to be evaluated), the restrictions that the agency placed on that evaluation, and any general or specific procedural requirements or restrictions enforced by the procuring agency. Remember that the debriefing process is an opportunity for agency officials to provide you with an explanation regarding any assessed weaknesses in your proposal. Negative assessments of your proposal might be due to a perceived lack of technical understanding. Rather than focus on how the agency should have interpreted your proposal, focus instead on the consistency of the agency’s evaluation methodology and whether its application was reasonable.
It is important to remember that the solicitation generally constitutes the entire universe in which the contracting officer must operate, and that an agency’s evaluation of proposals is confined to “the factors and subfactors specified in the solicitation.” [6] By way of example, if the source selection official identifies a weakness in your proposal, but the agency justifies that assessment against criteria outside the stated evaluation factors, you may be able to challenge that assessment in a protest. See Sterling Medical Corp., B-412407, Feb. 3, 2016 (“While agencies may apply evaluation considerations that are not expressly outlined in the RFP if those considerations are reasonably and logically encompassed within the stated evaluation criteria, there must be a clear nexus between the stated and unstated criteria.”).
PREPARE A LIST OF QUESTIONS
Prior to debriefings, contractors should prepare a list of questions. Focus on identifying questions that you would like to have answered. For example, contractors should craft questions not only relating to the debriefing requirements provided by FAR Part 15 but also relating to what aspects of your proposal the agency disliked (i.e., structure, presentation, and content). Undertaking this important task can guide your discussion during the debriefing and can strengthen your ability to elicit information from agency officials even if some of your questions go unanswered.
The Debriefing
TAKE NOTES
Contractors should bring someone to transcribe the agency’s comments at the debriefing. It is not always the case that a contractor will be given slides or narrative summaries during the debriefing. Therefore, contractors should bring a diligent note taker to create a written record of the debriefing. A documented record is important because it could serve as a critical tool during your post-debriefing discussions, especially to those relating to how you can improve future proposals. Additionally, if you discover flaws in the procurement, a written record will help your legal counsel understand precisely what information the agency conveyed during the debriefing.
CLARIFY
Contractors should be prepared to seek clarifications from agency officials when the answers to your questions seem vague or incomplete. Sometimes agency officials interpret the enumerated requirements under FAR Part 15 as the only information that they are required to share during the debriefing. Such a view disregards the FAR’s intent, which was to establish a baseline for debriefing information to encourage greater transparency. Regardless of what approach agency officials use, contractors should be prepared to seek clarity when needed.
For example, during your pre-debriefing strategy session, you may have been alarmed after determining that the solicitation required a price realism analysis and that the awardee’s proposed price was alarmingly low. You have prepared a list of questions to ask the agency at your post-award debriefing, however, agency officials respond to your realism question with, “the price analysis was consistent with the terms of the solicitation.” In this scenario, you may not have the answer you seek. This is the type of answer that demands clarity. Contractors should be prepared to refine their question, perhaps by inquiring whether the agency had performed the required realism analysis in accordance with a specific provision of the solicitation. Ideally, agency officials will openly respond to further inquiries but do not let nebulous responses deter you from your purpose, which is to gather information. While some procurement efforts contain errors, a relatively close-lipped debriefing is not necessarily evidence of a flawed procurement.
It should also be noted that thorough and transparent debriefings are not novel. Enhanced debriefings are examples of how agencies actively seek to improve the debriefing process. Enhanced debriefing techniques comport with the FAR to the extent that the requirements under FAR Part 15 are treated as a floor rather than a ceiling. Contracting officers who adopt this mentality during debriefings and provide comprehensive debriefings could also see similar results to enhanced debriefings–the reduction of costly, time-consuming litigation and the avoidance of unnecessary delays. Regardless of how much information the agency provides during the debriefing, contractors who are able to elicit detailed responses from agency officials will be in a stronger position during your post-debriefing analysis.
Post-Debriefing Analysis
Generally, contractors should gather the proposal team to discuss the debriefing as soon as possible. Return to the solicitation and your proposal to review the weaknesses or deficiencies assessed by the agency. Ask yourself whether the agency’s reasons for assessing your proposal negatively were reasonable in light of what you covered in your proposal and when compared to other aspects of the agency’s evaluation. In addition, make sure that any assessed weaknesses properly fit into the evaluation scheme required by the solicitation. Lastly, carefully reflect upon other aspects of your proposal that the agency officials disliked (structure, presentation, and content). While this is not remotely exhaustive of every issue that may be revealed in a debriefing, it can go a long way to giving you peace of mind and catching mistakes. Thoughtful analysis and willingness to modify your proposal can pay dividends in future procurements.
Takeaway
Debriefings are a great information-gathering tool that contractors should never turn down. [7] Contractors who approach debriefings thoughtfully and deliberately will be in a better position to take advantage of the information learned at the debriefing. Agency officials who understand the procurement from a contractor’s perspective can utilize that information to craft more robust solicitations, which would benefit the procurement community moving forward. Above all, contractors who thoroughly prepare for debriefings will be in the strongest position to take advantage the debriefing, both for future procurements and for formulating possible bid protest strategies. Remember, what is not measured cannot improve.
____________________
[1] See FAR 15.505 (pre-award debriefing) and FAR 15.506 (post-award debriefing); see also FAR 16.505(b)(6) (post-award debriefings for orders exceeding $5.5 million).
[2] See FAR 15.505(f) (prohibited disclosures for pre-award protests); see also FAR 15.506(e) (prohibited disclosures for post-award protests).
FAR 15.505(e) At a minimum, preaward debriefings shall include --
(1) The agency’s evaluation of significant elements in the offeror’s proposal; (2) A summary of the rationale for eliminating the offeror from the competition; and (3) Reasonable responses to relevant questions about whether source selection procedures contained in the solicitation, applicable regulations, and other applicable authorities were followed in the process of eliminating the offeror from the competition.
FAR 15.506(d) At a minimum, the debriefing information shall include --
(1) The Government’s evaluation of the significant weaknesses or deficiencies in the offeror’s proposal, if applicable; (2) The overall evaluated cost or price (including unit prices), and technical rating, if applicable, of the successful offeror and the debriefed offeror, and past performance information on the debriefed offeror; (3) The overall ranking of all offerors, when any ranking was developed by the agency during the source selection; (4) A summary of the rationale for award; (5) For acquisitions of commercial items, the make and model of the item to be delivered by the successful offeror; and (6) Reasonable responses to relevant questions about whether source selection procedures contained in the solicitation, applicable regulations, and other applicable authorities were followed.
[3] Interagency Alternative Dispute Resolution Working Group, Improved Debriefings, available at https://www.adr.gov/adrguide/ch08.html (last visited March 6, 2017).
[4] See generally, Interagency Alternative Dispute Resolution Working Group, AF Extended Debriefings, available at https://www.adr.gov/adrguide/08-afedp-why.html (last visited March 6, 2017).
[5] This maxim, or a similar amalgam, is commonly attributed to the great management consultant and leadership expert, Peter F. Drucker.
[6] FAR 15.305(a) (“An agency shall evaluate competitive proposals and then assess their relative qualities solely on the factors and subfactors specified in the solicitation.”).
[7] FAR 15.506(b) (“Debriefings of successful and unsuccessful offerors may be done orally, in writing, or by any other method acceptable to the contracting officer.”).
DISCLAIMER: This post is for informational purposes only and may be construed as attorney advertising in some jurisdictions. The information provided above is not intended to be legal advice and should not be construed or relied upon as legal advice. If you need legal advice, please consult an attorney.
