Investing in generative AI? Build an AI center of excellence first.

Less than a year after the release of ChatGPT, nearly half (47%) of surveyed companies already consider GPT’s root technology—generative AI—their top IT spending priority for the next 12 months.

For many of those early adopters, generative AI is their first dance with any form of AI. Which raises a tricky question: How can these companies ensure they get sufficient business value for their money?

For businesses taking a new dive into generative AI, the best way to hedge risk—and to avoid the generally high failure rate of any AI initiative—is to make a coinciding investment in an AI center of excellence (CoE), says Tom Davenport, a leading enterprise AI expert and bestselling author of “All In on AI” and many other books.

The core rationale behind an AI CoE, says Davenport, is to centralize AI resources, leadership, and expertise. To some, the idea may sound tedious and time-consuming when generative AI applications are seemingly so easy to set up and deploy.

But as Davenport explained in a recent interview with The Works, the “why” of building a CoE is as important as the “how” when it comes to wringing business results from core AI technologies.

Conversation edited and condensed for clarity.

Companies are obviously rushing into generative AI, and for good reason. Is there a rationale for tapping the brakes a little to make sure you’re building a solid AI practice at the same time?  

I think so, yes. When a capability is new, it’s more important to have it be centralized so that you have a critical mass of capabilities, people can learn from each other, and you can figure out what are the most important places to use this technology.

A CoE model allows you to explore questions such as, “where can AI add value in the company?” or, “which new business models and strategies can it enable?” From there, you can identify more achievable use cases for a wide range of AI.

Generative AI is an amazing breakthrough and something that every company should be looking at for content creation, such as easy-to-read reports or marketing copy. But the vast majority of organizations have other types of tasks they need to perform with AI.

If you’re a bank, you need to assess a customer’s credit before you approve a loan. If you’re a company with a salesforce, you need to make sure your reps are spending their time wisely, so you create a model that can predict which sales leads are most likely to close. If you’re a manufacturer, you want to use AI to predict when your machines need repair. It’s all AI, but it’s not generative AI.

Generative AI should be viewed as a powerful tool in the toolkit, but it’s certainly not the only tool.

For a company that is still new to deploying AI, what are some critical first steps to setting up a center of excellence?

Bringing together the right team. You’re going to want a C-level leader. One new role that has emerged in the C-suite is the chief data and analytics officer. Some organizations, including the U.S. Department of Defense, have a chief data and AI officer.

You’re also going to need data scientists, obviously, and data and machine learning engineers who can help scale up these applications and integrate them with the other systems around the company. 

I’m a big fan of a role called data product manager. These are people who manage the product, the identification, development, and deployment of AI-based offerings for external or internal users. Data scientists are great at creating models, but they’re not so good at managing change, building trust with stakeholders, or upskilling people. Data product managers can do all that.

Read also: AI is poised to solve the tech overload problem

What about leaders of business units considering generative AI who are itching to use it? If you’re going to be deploying AI in hiring or finance, how should you work with a CHRO or CFO?

I’m a big believer that you start with a centralized group, then build outreach to other parts of the organization. The CoE is the hub, with dotted lines to the spokes. Procter & Gamble did this years ago with their AI efforts. They had a centralized group of analytics people in their CoE, but day to day, they were mostly embedded within business units.

At P&G, the business leader said, “Great, we love these analytics people. The only problem is I want them reporting to me.” So they moved the solid line to the business, and the dotted line was to the CoE. That proved to be an effective organizational structure for them.

Are there new models for AI CoEs emerging, or is an AI center of excellence from five years ago going to look the same as one you’d see today?

The big change now is that these new centers are trying to industrialize AI and make it not an ad-hoc labor-intensive activity that it was in the past.

AT&T is a great example because they’ve concluded, as I have, that the way to large-scale digitization is not just to rely on professionals. They have a very active democratization effort underway out of their CoE with over 500 courses to teach people how to do machine learning and AI. They’re just one of the earliest adopters of this broad-scale democratization effort driven by a center of excellence, because they know that professionals aren’t going to be enough. They’ve got to enlist the amateurs.

Why is that so important for early adopters of generative AI?

Because there’s hardly any people who know how to build generative models from scratch. Most are employed by Google or OpenAI and startups, so you’d have a hard time attracting them. But if you have some data science talent or even people who are not trained technologists but are motivated to grow their skills and have particular domain knowledge, they’re going to be able to learn enough about generative AI to apply it to a particular problem for business domains. 

In terms of new roles like prompt engineers, who can train and tune these AI models through both natural language and coding prompts, it’s really important to have a mixture of data science and domain knowledge. If you’re in a law firm, they have to know the law as well as how these generative systems work. You may have to find some people who have the domain knowledge and are interested enough in generative AI to learn about it and do prompts. With a center of excellence, you can grow the talent from the inside.

We want to hear from you! Please send us your feedback, and get informed about exciting updates from The Works. Drop us a line: theworks@freshworks.com.