Highlights from Craig Milley Interview on the Cayman Islands 2020 Vision Case Study

01 May 2017

Category: 20/20 Vision Interview Paul Emmanuelli

For the May 2017 case study on 2020 Vision: How to Relaunch Your Tendering System, Paul Emanuelli travelled to Grand Cayman Island to interview Craig Milley, the Director of Procurement for the Cayman Islands. Milley, who previously led Lethbridge, Alberta, to Summit’s 2013 Procurement Leadership Award for his successful rollout of Negotiated RFPs, is now leading the deployment of Cayman’s new procurement system. In a wide-ranging interview, Paul and Craig discuss how to leverage smart procurement technologies to train end-users on new procedures that comply with international procurement standards and touch on the following topics:

  • How the Cayman Islands implemented UN Model Procurement Law standards, including the concurrent and consecutive negotiated RFPs that have now been formally adopted under Canada’s new trade treaties, to address past compliance concerns raised by the Cayman Auditor General;
  • Why it was essential for a small procurement department to leverage smart tendering technologies to train end-users in achieving compliance with international standards;
  • How Cayman uses an automated RFX drafting system to help guide end-users to the appropriate tendering format and coordinate the RFX drafting process in a decentralized environment;
  • Why the Cayman Islands implemented electronic bid submissions and how this has helped them accelerate tendering cycles, reduce their carbon footprint and avoid being a late bid case study;
  • How the Cayman Islands responded to the risks associated with consensus scoring as revealed in recent high-profile U.K. litigation, and implemented enhanced consensus scoring methods through an electric bid evaluation platform;
  • Why the Cayman Islands’ electronic auctions pilot project, which led to 20% cost savings, should serve as a model for implementing the electronic auction protocols recently adopted under Canada’s new trade treaties.

Highlights from the interview are summarized below:

Part 1: Leveraging Technology to Deploy New Tendering Templates

The Auditor General Report

Craig Milley’s journey in implementing international procurement standards in the Caribbean began with a report by the Cayman Islands Auditor General. As Milley explains, the Auditor General concluded that “the government had lost control of the public procurement process.” In the aftermath of that report, Milley, who was the head of the Lethbridge procurement department at the time, was hired by the Cayman government to help lead their procurement modernization effort.

Updating and Implementing New Rules

Milley’s Cayman initiative had two components. First, he needed to update the procurement governance rules. Second, he needed to quickly implement those changes. Milley drew on a broad range of international standards, incorporating elements of the UN Model Procurement Law, while looking at other jurisdiction within the Caribbean, as well as governance structures in Canada, the U.S. and the U.K. As he explains, Cayman became “an experiment to bring in the best aspects of many different jurisdictions without also inheriting lots of the problems that are part of the governance structure in other parts of the world.”

Leveraging Technology

Given his time and resource constraints, Milley also determined that technology was the critical link to quickly implementing the new procurement system. However, he is careful to point out that technology is not a substitute for knowledge of the procurement system. The key to success was selecting smart procurement technologies that help accelerate the implementation of proper procurement formats and procedures.

Leveraging the UN Model Procurement Law

Milley notes how the UN Model Procurement Law served as one of the key benchmarks for updating and expanding the range of tendering procedures and templates. “One of the things the Auditor General mentioned was the fact that we were using one type of procurement process for every type of acquisition. We had a one-size-fits-all approach. Clearly it didn’t work. One of the things we wanted to do was make sure we introduced all the different procurement methodologies that were out there. We wanted them based in international norms.”

Legitimizing Advanced Tendering Formats

As Milley explains, “the UN Model Law really helped us get recognition that these were legitimate tools. Whether we’re talking about the use of concurrent negotiations, a competitive dialogue, whether we’re talking about use of electronic reverse auctions, there were other methods out there that can and ought to be used in projects. That was one of the nice things we took away from the UN Model Law. We actually built that into our new governance framework.” This is particularly relevant in Canada since Canada’s new trade treaties have adopted many of the same international standards.

Adapting to Local Conditions

However, those international standards must be adapted to local conditions. As Milley observes, their tendering templates had to be “Caymanized” so that “people can recognize them as being our own but underneath it we can still have the international standards. That was something we always wanted to make sure, it’s not as simple as bringing another jurisdiction’s systems and processes and transporting them wholesale without regard for any kind of local content.” As the Cayman experience shows, public entities need to harmonize international standards with their local needs.

The Gold Standard of Procurement Automation

Once the groundwork was put in place through a proper local integration of international standards, technology then served as the conduit for the rapid deployment of those new processes. By quickly implementing the 2020 Vision, the Cayman Islands set an international gold standard in the adoption of smart procurement technologies.

Buyer Beware of “End-to-End Solutions”

The Cayman case study serves as a better option for institutions that are considering high-cost “end-to-end” technology solutions that force local procurement systems into expensive, complex, and outdated technologies. These “end-to-end” initiatives inevitably fail since their generic applications lack procurement intelligence and require inordinate amounts of customization to adapt to international procurement standards and local needs.

Process First, Technology Second

By contrast, in implementing the 2020 Vision, with its integration of smart procurement automation, Cayman avoided this pitfall. As Milley explains, “my mantra has always been ‘the process first, technology second.’ You have to design your process and then the technology that enables that process comes secondary.”

A “Procurement-Centric” Approach

Taking a procurement-centric approach to technology may be counter-cultural to large organizations looking for an “out-of-the-box” solution to all their procurement needs. However, to properly work, the technological solution must first be based on a proper understanding of procurement procedures. Automating the procurement cycle cannot be driven through a generic and abstract set of programming norms since computer programmers are not procurement professionals. They lack procurement know-how. Starting with procurement intelligence makes the difference between success and failure in procurement automation.

Strategic Procurement

As Milley notes, “this discussion is part of the broader context when we talk about strategic procurement. It’s looking at procurement, but from a human standpoint. It’s about making decisions that not only help the process but also help the client, the customer, and the owner at the end of the day. It’s really thinking of it from a strategic standpoint rather than just thinking the software is going to solve our problems.”

Strategic Execution

Cayman has a relatively small procurement office within a highly de-centralized procurement operation. This presented significant challenges in the deployment of the new tendering formats. As Milley asks, “how do you ensure that people can carry out all of these different methods?” He solved this impasse by implementing the smart procurement technology contained in the Orbidder system, which guides end-users into selecting the appropriate tendering template and then coordinates the incorporation of content into the selected template.

The “95/5 Rule” of Strategic Procurement

The Orbidder system applies the “95/5 rule” of strategic procurement, which recognizes that 95% of procurement failures are caused by the failure to address the five core project design considerations. Project scoping, pricing structures, evaluation plans and contract assembly strategies must be clearly identified at the start of a drafting process since those four factors inform the fifth factor, which is the selection of the appropriate tendering format. With its interactive interface, the Orbidder system helps build project blueprints based on the 95/5 formula.

Using Automation to Train End-Users

As Milley explains, since his end-users are not procurement professionals, they need to be quickly trained on the selection of the appropriate tendering format. The ability of a software platform to take the end-users through a quick decision tree helped bridge this resource and knowledge gap. According to Milley, the Orbidder technology can “take someone who is not experienced in all the rules and regulations of public procurement, it can help them decide and can actually drive them to the appropriate format. That’s been a real game changer.”

Technology Expands Procurement’s Influence

Milley explains that the Orbidder system allowed his small office to exponentially expand its influence across the organization. “We have fourteen ministries and twenty-six public agencies that we deal with on a regular basis. We can only be in so many places at one time, and certainly automating this whole bid drafting process has been a tremendous tool. Not only does it allow us to teach and train, but it also allows the end-user to make those decisions, and we are still able to effect and influence the outcomes.”

The Missing Link for Full Implementation

In the Cayman Islands, the use of smart procurement technology made the difference in implementing a full range of tendering formats. As Milley acknowledges, “this is probably why the default for many years had been one type of procurement format, because you just don’t have the ability to train and educate everybody on the spot.” Smart procurement technology served as the missing link for Milley. “Right now we can reach all of our clients in all forty different public agencies and certainly that does allow us to be influential without having to actually be there.”

Coordinating the Drafting Process

As Milley explains, the Orbidder system goes well beyond template selection and also enables the proper coordination of the entire RFX drafting process since “the system also allows us to keep visibility of the project so we can keep going in, double checking if things are moving along.” This allows him to “quickly check in with a client on a regular basis as they progress through the various stages” of the drafting process. Milley notes that this has helped avoid the dreaded “drive-by RFP” that comes into procurement departments late in the process without any warning and contains serious content flaws.

Clients Set Their Own Schedules

As Milley states, “the beauty of Orbidder is that the client drives the schedule. One of the first things they are prompted to enter into the system is their expected issue date. That’s their decision as to what date they put in there.” That issue date serves as the hub for the entire drafting process and informs internal progress deadlines. In Milley’s experience, “the software is an amazing tool because once that date is entered into the system, the project team are constantly reminded of their own schedule.”

Greater Visibility

Milley also finds that the Orbidder system allows him to maintain control over the drafting process since “you can go in and actually see how they are doing. You can go in, take a quick look, you can contact the client just to find out if there’s something happening beyond their control. It gives you total visibility. I’ve often said I wish this software was around for the past seventeen years because it would have made my life as a procurement officer a lot easier. That visibility and the ability to see inside the project is priceless.”

Automating Version Control

The Orbidder system also enables the rapid assembly and review of working drafts while maintaining version control. As Milley explains, “Our clients love the ability to run different versions, especially with the larger teams. The project manager even has the ability, as simple as toggling a button, to lock out the drafters. If this was a larger team, you might want to run a version and just see what the draft looks like. You can toggle out the drafters, run the version, take a look at it, and then toggle them back in. The editing process continues so you won’t be stumbling over each other in terms of who’s got the latest version.”

Coordinating Concurrent Drafting

This Orbidder system also allows the project leads to assign different drafting roles to different members of the team so, as Milley explains, “someone can work on the legal agreements, somebody can be working on the pricing form, somebody can be working on evaluation criteria. All of those different aspects can be delegated to a member of the team and the project manager is ultimately going to be coordinating that group effort.” 

Hub and Spoke Assembly

Rather than waiting in sequence to start drafting, each member can draft concurrently in a coordinated way. As Milley states, with the system, “you truly have the ability for work to be going on concurrently so that everyone is working in their own space on their particular piece but its all housed and is ultimately visible within one project.” When compared to the manual use of templates, this hub-and-spoke assembly, with the automated tendering templates in the middle of the control system, helps break the classic logjam of a traditional RFX drafting process.

Intuitive System Requires Little Training

Milley notes that the end-users on the Oridder system were able to immediately use the software to help them with the new tendering templates. As he explained, “the functionality is very intuitive when it comes to taking a client through for the first time who has never seen the software before. We give them a brief orientation, and help them set up the initial structure of their project, and then they go away to start working on it.”

Instant Deployment

Milley can then monitor the progress on the drafting, stating that “we’ll communicate with them on a regular basis but its not a large-scale rollout in terms of training and turning everyone into an expert. The ability to use the software, you can use it right away. You will get better, you will get quicker, you’ll get faster as you do more projects within the system but it’s certainly intuitive enough right out of the gate that you’ll see benefits immediately.”

Bridging the Resource Gap

Milley explains that using the Orbidder system allowed him to deploy Cayman’s new templates with far fewer resources than in a prior manual deployment. As he recalls, “My experience with the manual templates goes back to 2011 in my prior life with the City of Lethbridge in Alberta. The difference in Lethbridge was that my team were fairly seasoned professionals, a bigger team and more focused on procurement issues on a day-to-day basis.”

Automatic Assembly

Milley states that using the Orbidder system allowed him to quickly deploy Cayman’s new templates with far fewer resources than in his prior manual deployment. As he explains, “The difference here is that we’ve been able to do the transition almost as quickly with people who are not experienced public buyers. It’s night and day.” Furthermore, once the end-client departments have finished preparing their content, there is no longer a need to manually assemble the pieces together into the final documents.

Smart Technology for Rapid Deployment

As Milley explains, “The system does all of that. Orbidder will synthesize so if you bring in a pre-existing large technical document that you wanted to incorporate into the RFP, it’s a matter of attaching the file. The system will take it in, will number it, will actually assemble it as part of the final document in the right place.” By launching the Orbidder system, the Cayman Islands has implemented international standards while taking its document drafting to a higher level of speed and precision.

Part 2: Using Bid Evaluation Technology to Evaluate Proposals

The Challenge of Group Evaluations

The second part of the 2020 Vision Cayman case study focuses on the use of automated bid evaluation technology. The procurement industry has long been log-jammed with in-person consensus scoring meetings that drag on for days and even weeks, with evaluators debating every line item of every proposal in excruciating detail, negotiating the distinctions within their evaluation committees between 8.1 and 8.3 out of 10, while someone tries to keep notes to record what is occurring during that process.

Lawsuits Waiting to Happen

In recent years, there have been a series of lawsuits, including the 2016 case of the year out of the U.K., which found fundamental flaws in group evaluations. As Craig Milley explains, Cayman needed to implement a more efficient approach that did a better job of mitigating those legal risks. Cayman is using a bid evaluation platform called Bonfire, which, Milley notes, helps “speed up and ease the pain of those large group consensus meetings. The system is predicated on individuals doing the scoring, the system capturing the scoring, and amalgamating the scores from all of the evaluators.”

Lightening the Load for Evaluators

As Milley notes, the Bonfire evaluation platform has helped to streamline and enhance the defensibility of the evaluation process. “The folks can read the proposals, access the documents, do their work where they wish. I think that our friends at Bonfire show that 25-30 percent of all evaluations are done outside of business hours. That’s probably always been the case, people would take home large binders or stay at the office late to read proposal. This software actually allows you to free up time and to have easier access. There’s a lot of wins for the evaluator.”

Enhancing Accountability

In addition to efficiency, the Bonfire tool also helps to promote accountability. As Milley states, “the system is providing an audit record of everything that happens so that the auditors are reassured by the audit trail. It works both ways: it keeps our evaluators on point and on target, with the system auditing every step along the way, and it really helps the individual evaluators on the team.”

Shielding Against Interference

The system also helps shield evaluators from inappropriate interference. As Milley observes, “Sometimes evaluators have to deal with people who aren’t on the team trying to influence how they score, to influence the outcome, so I think that this software really has the ability to protect the evaluators. As a procurement official that is certainly my goal. One of the objectives is to make sure that the individual scores are recorded and respected and not discounted, and ultimately to make sure that we have a defensible decision, we have a reasoned decision, and we have documented decision.”

Implementing Enhanced Consensus Scoring

Bid evaluation technology also helps with the implementation of Enhanced Consensus Scoring, which is a method that has been developed to deal with the litigation risks relating to traditional consensus scoring. From an efficiency standpoint, the technology allows you to immediately identify the outliers in the evaluation scores so that you can quickly hone in on areas where potential mistakes could have an impact on the final rankings and thereby create legal liabilities.

The Cayman Example

As Milley observes, this is something that really helps speed up the process. Rather than going line-by-line, bidder-by-bidder, evaluator-by-evaluator in an outdated manual approach, the system can automatically identify the outliers based on the variation tolerance set by the evaluation team.” While Milley acknowledges that you could identify the outliers manually using spreadsheets, “the difference is the speed and how fast you can identify it. You may not even need the entire group together, if two people were the source of the disagreement, you could have that conversation quickly.”

Beefing Up Debriefing Practices

Bid evaluation technology can also help organizations with their debriefings, which is a live issue in Canada since the new trade treaty rules will require the disclosure of more details about the evaluation and scoring of proposals. In fact, the debriefings under the new Canadian procurement rules will no longer be voluntary, but will now be a mandatory first step in a potential bid dispute process. As the Cayman case study illustrates, from the standpoint of internal governance, having a tool that enables automated evaluations is no longer a luxury, it is a minimum requirement for meeting international standards.

Going Green with Paperless Bidding

To enable bid evaluation platforms, institutions need to migrate to paperless bid submissions since proposals need to be loaded into the system in electronic form. As Milley notes, going paperless also has a positive side-effect from an environmental perspective, “Year-to-date we stand at over 12,000 pages of paper avoided.” Given the amount of paper saved in a smaller jurisdiction like the Cayman Islands, one can only imagine the amount of paper that will be saved in a country as large as Canada once all institutions adopt paperless bidding.

Accelerating the Tendering Cycle

Furthermore, under the new Canadian trade treaties, using electronic bids permits faster closing times. As Milley explains, “For us, it’s been significant, as a small jurisdiction, small island in the middle of the Caribbean, a lot of our suppliers are bidding from overseas. In the paper-based world, they have to stop work a week to ten days before the deadline, print that bid up, entrust it to a courier, fly it over, and get it here before the deadline. Now they can bid to us electronically in a secure platform and they’ve gained an extra week to work on it and saved printing costs and the shipping costs.”

Making Room for More Efficient Evaluations

Finding space to camp out for consensus scoring is getting harder and harder given organizational time and space constraints. With bid evaluation software, evaluation teams no longer need to be taking up valuable meeting space while they are trapped in a room with a sky-high pile of binders in consensus scoring sessions. Technology has resolved this paper-based bottleneck since evaluation teams can meet remotely to review proposals once they have scored those proposals individually.

Better Monitoring of the Process

As Milley explains, “we have total visibility into the process as it’s unfolding. We can see the progress of the evaluators, we can see their scores as they start to unfold. We can see the aggregate scores start to build up. All of that gives us visibility. Plus we have conflict of interest disclosures, evaluators don’t get access to the bid submissions until they sign the conflict of interest disclosures – we’ve configured the system so that’s a mandatory – access to the system is controlled from our central office so that we ensure that the right people have access to the evaluation.”

Easier Record Management

As Milley confirms, from a legal and audit perspective, a bid evaluation platform is superior to paper-based tendering since the ability to digitally control a document significantly reduces the risk of lost, misplaced and misdirected records. By contrast, manually managing document control is far more challenging, especially for smaller procurement departments. As Milley warns, storing paper proposals is also becoming an issue. By using bid evaluation software, the taxpayer will no longer have to pay to archive rooms full of old paper proposals.

Thumbs Up by End-Users

Bid evaluation technologies are also popular with end-users. As Milley explains, “the Bonfire folks are really great at surveying. The overwhelming response from evaluators was ‘Why weren’t we doing this all along?’ I have not yet had any negative feedback from an evaluator. The next time they are on a project, they are knocking on my door.” When combined with the Orbidder drafting system, the Bonfire bid submission and evaluation tool has helped accelerate Cayman’s tendering process, bringing greater speed and precision to their overall procurement cycle.

Part 3: Electronic Auctions

Automating the Pricing Form

The third and final segment of the Cayman Islands 2020 Vision case study explores electronic auctions, a technology that has evolved to allow the procurement industry to move well beyond the traditional one-shot approach to price submissions. With electronic auctions, the price component is transformed from a static form into a digitized platform where price evaluations are automated to enable the collection of literally hundreds of price refinements that are assessed and factored into proponent rankings in real time.

Driving Cost Savings

Milley explains that the impetus for electronic auctions in the Cayman Islands was the need to save money. As he states, “in the wake of economic downturn in 2008-09, the government found itself having to go with cutbacks and impose austerity, so there was a real driver to find a way of reducing the overall cost of government. The task given to me was to identify potential savings targets and to come up with mechanisms to create savings. This was a task that I wanted to do in a disciplined way. I did a lot of research and over and over I kept coming back to electronic reverse auction as a possibility.”

Electronic Auctions a Widespread Practice

The Cayman Islands recently completed a successful pilot project with EASiBuy, a U.S. platform provider, and is looking at implementing a broader program. Milley notes that Canada remains an outlier with its failure to implement electronic auctions. As he states, “Coming from Canada, we don’t have much experience in this area, but what I found in my research was that it is very common in the United States. In the U.K., this was a normal course of business, and as I started looking further afield, I found that in other jurisdictions, even in Central and South America, it’s a common technique.”

Pilot Project Hits the Mark

The Cayman Islands’ pilot project successfully hit the industry-standard target by achieving nearly 20% cost savings in its initial deployment. As Milley summarizes, “in the auctions we’ve done to date, we achieved a savings of just over 19%. It wasn’t a surprise. We were expecting to be in that 20% range, that was what the literature and studies had shown of what the initial savings should be in our first use of electronic reverse auctions. We found that we were right in the target zone.”

Dynamic Price Competition

As Milley explains, electronic auctions are far more efficient than traditional price negotiations and make for a more dynamic process. By way of example, in his most recent process, proponents were initially scored on technical factors to identify two finalists for the electronic auction stage. During the electronic auction, Cayman received 1400 bids in a five-and-a-half-hour period. The lead changed about forty times as the finalists competed on price.

Protecting Confidential Bidder Information

Milley also points out that bidder pricing information is protected in the online platform. While the bidders see their ranking in real-time relative to their competitors, they do not see the pricing. Bidders push their price into the platform. The platform then absorbs the pricing and translates it into a horse race, and that’s what the bidders see. They never see their competitor’s actual pricing. As Milley notes, “the supplier only knows their position, their positional rank” so there is no issue regarding the disclosure of confidential pricing information to competing bidders.

Other Scored Factors Still in Play

Milley goes on to note that factors other than price can still inform the overall ranking. “The other part that is important is that we are still doing the pricing event as just one portion of a traditional RFP process. We’re still evaluating quality and experience and scoring them in a very traditional way. The only real difference is pricing, how we evaluate pricing.” When compared to one- or two-staged tenders, manual negotiations and face-to-face meetings, electronic auctions allow organizations to leverage greater price competition. It is self-evident that there is no way to manually accomplish the same result.

Canadian Adoption

While traditionally there have been very few entities in the Canadian public sector who have expressed interest in using this approach, no organization can justify spending 20% more than it needs. Canadian institutions will be under increasing pressure to adopt this cost-saving technology. Now that electronic auctions are formally recognized under Canada’s new trade treaty rules, it is only a matter of time before they find their way into the mainstream of the Canadian public procurement system.

Picking the Right Project

However, Milley warns that electronic auctions should be used strategically and selectively. As he states, “it is not the methodology that you use for everything.” In fact, selecting the right project is a key to success. For example, Milley notes that at the City of Los Angeles “They look at a 1000 contracts per quarter, of those about six to ten get reverse auctioned. It’s a very small percentage but going back to the strategic procurement aspect, key commodities, key services, it’s an invaluable tool.” In the right project, electronic auctions can make a big difference.

Adding Electronic Auctions to the Toolbox

The Cayman Islands plans to run its next electronic auction for pharmaceuticals, a high-spend category with a large impact on the economy. Milley recommends that Canadian organizations interested in electronic auctions follow the Cayman example and launch a test-pilot. As he states, “it’s okay to be skeptical but don’t discount the methodology because it has not been done in Canada or in your own community. Approach it from a strategic standpoint, be open minded about it, and accept the fact that it can be a very important tool, one of many tools in your toolbox.”

Rebooting the Tendering Cycle

Along with the Orbidder drafting system and the Bonfire bid evaluation tool, electronic auction platforms complete the trilogy of smart procurement technologies that helped the Cayman Islands set the gold standard for implementing the 2020 Vision and automating their tendering cycle. Canadian public institutions can draw from this case study to help chart their course as they navigate the emerging challenges of Canada’s new trade treaty regime.