Multimodal Machine Learning to Streamline Call Center Interactions

In summer 2024, I worked as a software developer at Amplify, RBC’s flagship early talent program. While there, my team and I developed a prototype of OCTO, a product which streamlines complex call-center interactions and saves time for clients and advisors. Now that our our patent application is publically available, I’ve decided to write a short blogpost about what we built, and share a little bit about the experience.

DISCLAIMER

This short blogpost intentionally omits many details to ensure that I uphold my NDA with RBC. All content here is described in less detail than our publically available patent application.

A Problem with Traditional Call Center Flows

In a typical call-center workflow, automated voice assistants are useful, but limited. After a client describes their issue, straightforward requests can be mapped onto a decision tree and are handled entirely within that system, without ever needing an advisor. For complex requests, the voice assistant reduces the client’s description of their issue to a broad call category and then places them in a queue. After the wait, the client is connected to an advisor, who takes over from there.

What is missing is an in-between mode. Once a call is headed for an advisor, the voice assistant stops being an active part of the workflow. It may have helped route the call, but it does not continue to help the client make progress while they wait, even if a portion of their request is self-servicable.

The Opportunity

Instead of asking the voice assistant to fully solve the problem or step aside completely, there is a third option: keep helping after the client has already been placed in a queue and is waiting for an advisor. Actualizing this idea would require building a system which identifies likely first steps of the call, and allow the client to self-serve the simplest of those tasks during time that would otherwise be spent idly waiting.

OCTO: the Omni-channel Triage Optimizer

While the client is in the cue, OCTO combines the client’s spoken responses and their client profile to predict actions that the client is most likely to need help with. Combining multiple sources of data in this way is the cruicial insight which enabled us to identify concrete actions to be taken, rather than broadly predicting the reason for calling.

Once these actions are identified, OCTO separates them into two groups: ones the client may be able to complete on their own, and ones that require an advisor.

If there is a simple self-serve step available, OCTO invites the client to complete it while they’re still waiting in the queue, by sending a link to their device. At the same time, the call remains in the advisor queue, so the client does not lose their place while interacting with the self-serve option. This lets them get started on solving their problem while waiting for an advisor.

Once a client is connected to an advisor, the advisor gets a status update with predicted action information, including which steps the client pre-completed while waiting. Even if the client elects not to use OCTO, the advisor can get up and running more quickly, as they need to ask fewer probing questions in order to determine which tasks they should start with.

Pitch Competition

Amplify event photo 1
Amplify event photo 2


We pitched our product at AmpExpo, the final celebration and competitive showcase of RBC Amplify. After being selected for the finals, my team pitched to a panel of Chief Executives, who awarded us the coveted Industry Disruptor Award! This achievement affirmed the innovative nature of our product, and its ability to transform the customer support-seeking experience while saving RBC millions of dollars annually. This whole thing was truly a surreal experience.