How language A.I. transformed Bloomberg’s business—and may change yours too

A few weeks ago, I promised more from my conversation with Gary Kazantsev, head of quant technology strategy in the office of the chief technology officer at financial news service Bloomberg. Previously, he ran machine learning engineering there. (Full disclosure: I worked at Bloomberg News before Fortune.) Gary, who also teaches courses on machine learning at Columbia University, is a font of knowledge about the current state of artificial intelligence in business.
You might know Bloomberg from its news service, which includes a cable television and a radio channel, as well as a news wire and website. But the company makes most of its money from financial data. Financial institutions subscribe to the “Bloomberg terminal”— once a dedicated piece of hardware, but now a software package that can be accessed online. A subscription gives users access to an immense range of data about stock, bonds, commodities, currencies, as well as the ability to search, parse, and graph that data. There is so much news and data available “on the terminal” that a perennial problem for Bloomberg is that most its customers only ever use very limited subset of functions. Compounding this problem is the fact that until recently users had to memorize obscure three- and four-letter codes to run the terminal’s functions. I remember that as a new employee at Bloomberg in 2011, I spent an entire week in training just to learn the rudiments of using the terminal.
Kazantsev was keen to show me how Bloomberg has, in the past few years, used new capabilities in natural language processing (NLP) to transform how customers find content on the Bloomberg terminal. And the way Bloomberg has deployed NLP holds lessons for other companies hoping to use NLP to change how customers interact with products and the business a whole.

Thanks to advances in NLP, a Bloomberg user no longer needs those obscure codes. She can simply write in the command line, “Find all the U.S. corporate bonds with a yield greater than 4%, a rating better than BBB, and a maturity before 2025,” or “Who are the top five holders of Apple stock?” and the system will provide the answer. Before, getting this information required a time-consuming, multi-step process involving several different commands and, in the case of screening searches for stocks or bonds, filling in fields in a database query interface. The new system also auto-completes as a user is typing, suggesting possible queries—much like the Google-search bar does. This allows a user to discover options—such as a type of analysis or a graphing option—they otherwise may not even realize was available.
Not only have these new NLP capabilities helped Bloomberg’s customers get more out of their product. They have also helped improve how the company’s customer service reps provide answers to these clients. The “question answering” NLP A.I. is used in about 50% of its customer service calls now and in more than a third of cases, the A.I.’s top suggested answer is the one the customer service rep recommends to the customer.
While there’s been a lot of buzz about ultra-large language models, Kazantsev says that Bloomberg’s natural language question-answering functionality is not built from a single ultra-large language model. Instead, it is a modular system using many different components including “a query intent model” that tries to predict which function the user wants to run and a “semantic parser” that tries to classify the relationship between the words in the sentence and then label those words as either entities (essentially proper nouns of some kind) or attributes (is it a date for example?) And then there is a module that Kazantsev says kind of runs that semantic parser in reverse to make the auto-complete suggestions. For some aspects of what Bloomberg does, it uses a “large-ish” natural language model that has been fine-tuned on financial text.

Why doesn’t Bloomberg use an ultra-large language model, of the sort that OpenAI has built with GPT-3? Well, when you get models that are that big—taking in more than 100 billion variables—it takes too long to run each query, Kazantsev says. Each answer would take seconds; Bloomberg needs to generate answers in fractions of a second.
Kazantsev says he’s fascinated by ultra-large language models from a research standpoint—they do seem to have really incredible, emergent properties (such as explaining the logic of jokes without being trained to do so)—but for many practical, business tasks, the problem remains, “what do you do with them?” They are simply too unwieldly to be practical—at least for now.
There are some key lessons for other companies here: The NLP revolution is real and can be transformative. Customers increasingly want to interact with technology in natural language—not complicated codes, or, for that matter, a series of drop-down menus and database fields.

This is true for computer programming too—one of the most impactful things to come out of the NLP revolution may be A.I.-enabled software that lets a person specify in natural language what they want a software program to do and then the A.I. will write the appropriate code. But modular systems made of smaller components are more likely to be how most businesses bring natural language understanding to customers, rather than ultra-large language models.

Finally, before we get to this week’s news, I want to wish a fond farewell and good luck to my co-writer on this newsletter, Jonathan Vanian, who is leaving Fortune after seven years. Look out for him popping up on your TV on CNBC.
Also, a quick correction: In last week’s newsletter, I misspelled the name of Omilert, one of the companies that makes gun detection software. I regret the error.

Jeremy Kahn
@jeremyakahn
jeremy.kahn@fortune.com

This story was originally featured on Fortune.com