Bio: In the early 2000’s, Arnab discerned that the information explosion created an unprecedented opportunity for value creation, and that a firm that combined superior talent with the techniques and technology required to distill insights from massive data reserves could deliver dramatic performance improvement. Accordingly, he founded Opera Solutions in 2004 and has since guided the company in providing rapid, significant and sustained profit improvement to leading global organizations. Prior to Opera, Arnab founded and sold a number of other companies, including Mitchell Madison Group, which achieved a recurring revenue base of $275 million within 4 years, and Zeborg, a business intelligence software company. He began his career at McKinsey & Co., where he was a partner; he also served as a partner at A.T. Kearney. Since 2004, he has worked with the Bill & Melinda Gates Foundation on its India HIV/AIDS initiative, employing private sector approaches to help oversee the disbursement of $400 million in grants toward HIV/AIDS prevention. Arnab earned an MBA from the Harvard Business School and is the author of a number of research publications, including Aggressive Sourcing: A Free Market Approach (Sloan Management Review) and Taking Risks to Win.
Company Overview: Opera Solutions has created one of the world’s most advanced, end-to-end platforms for using analytics to turn big data flows into profit and advantage. The company has built one of the world’s premier centers of predictive analytics and machine learning science, with 180 scientists and extensive libraries of proprietary signals and models. Its Vektor™ platform extracts powerful signals and insights from massive amounts of data flow, and then streams analytically enriched guidance and recommendations directly to the front lines of business operations—all without major IT or infrastructure impact for customers.
NTT Com: Why did you found Opera Solutions?
Gupta: We started the company in the financial services industry about eight years ago. It was becoming quite obvious, even then, that what we now call big data was going to be a transformational force for banking and for financial services in general. I had always known there was power in this because following the Internet revolution, data increasingly moved to the center of any business equation. I saw room for a company that specialized in big data analytics. We began as a consulting business, which gave us the ability to build our solutions and Vektor platform over time while working in partnership with very large organizations.
NTT Com: What kinds of gains had you seen companies achieve by analyzing this data they now had at their fingertips?
Gupta: If you look at where big data arose, there are a couple of areas. One is on Wall Street with the whole quant revolution. The other was credit because, increasingly, machines made credit decisions automatically. We started our work in consumer credit when people were using judgmental methods. When we compared judgment to machine-based methods, we found that machine-based methods routinely beat judgment-based methods by anywhere from 15 to 40 percent. This provided enormous leverage for us to create a systematic method to extract matter from data.
The problem with human judgment is that even a topic expert can only use a few pieces of information to form an intuitive or knowledge-based judgment. A machine can look through hundreds of thousands of pieces of information. And a machine does not have the cognitive biases that people have. That, for me, showed the power of what big data analytics could do. Over time, I came to understand that the machine method, even when pushed to the ultimate, was still lacking because it lacked human judgment. So increasingly, our solutions are based in advanced science, but we always ensure that they are transparent and can be expanded and enhanced by human judgment.
NTT Com: Tell us how your Vektor platform differs from other big data analysis options.
Gupta: If you look at the big data activities occurring now, they are largely in two areas. One is processing very large amounts of data, through Hadoop and similar platforms. The notion here is that because we have big amounts of multifaceted data, we need a bigger dam to process the data. We have been data technology agnostic. We extract, clean and process data through whatever technology you want. In the world of big data, you confront a multiplicity of environments. So our stack, at least on the ETL layer, is oriented around the ability to work with any data solution out there. On the other extreme you have all of the visualization companies, like Palantir. These companies believe that with big data, you must create transparency so that you can integrate information easily, allowing people to peruse the data and understand what is what.
What we offer is in the middle. We have created a new layer of the stack, which we call Signal Generation Management System (SigGMS). For us, the critical part of any big data situation is signal extraction—what signals are embedded within the data? When I say “signals,” I mean a piece of data that has predictable, very high descriptive meaning. The reality is that most big data is just big noise; most data is not useful. We have about 180 PhD-level machine-learning scientists within the company. They specialize in extracting useful, predictive and descriptive signals from any data source, whether structured or unstructured.
NTT Com: What is the importance of these signals in a business setting?
Gupta: For example, in the world of fraud, if somebody makes three or four consumer electronics transactions in a short amount of time, this usually means that fraud is possible. That is what we would call a “signal.” There is an enormous amount of science behind figuring out what these signals are. Sometimes they are very opaque. But the outcome is that we create libraries of signals in different domains, which we then repurpose to manage specific situations. SigGMS is what allows us to extract these signals, and then deploy them in production settings. This is the heart and soul of our stack.
NTT Com: Why is it important that your system is offered as a service?
Gupta: There is no such thing as a static algorithm. Algorithms evolve through use. When we put in a new solution, we also observe how users are using it. The reason for offering it as a service is not just because it is cost effective. Here, it is business effective. We need to learn from actual human behavior whether these algorithms are doing what they were designed to do. Often you find that they mostly work, but there are gaps in our knowledge. The feedback loop allows us to fill in these gaps and refine these algorithms continuously. At the core of our Vektor stack are these two things—SigGMS and this learning loop.
NTT Com: Do you feel like the idea that not all data is valuable data is a new concept for those caught up in the promise of big data?
Gupta: On a generic level, people sort of get it. But the reality is that their behavior is not in synch. The whole field of analytics lacks structure. If you think about it, looking at an Excel spreadsheet is analytics—a human being analyzing data visually is analytics. Business intelligence tools are analytics. So the field lacks segmentation or boundaries. We look at analytics much more narrowly. We define analytics as the underlying patterns in a human or market behavior, which can be extracted from data. This usually is not something that can be put in a spreadsheet to show trend lines.
What big data really represents is, for the first time in human history, we can see the externalization of human experience. Previously, when people did things, these things would get lost in the ether. Now, with people always telling you where they are and with appliances and sensors always tracking you, we are pretty much in the world of “Minority Report.” We leave an audit trail of ourselves everywhere we go. We can look outside actual experience, not relying on human experience and memory, but viewing the actual audit trail. The map of this audit trail shows us that there are only two or three degrees of separation between any two human beings. This is not well understood right now. Right now, people tend to extend methodologies from the past. They think if they just make the information available and people can look at it, then that is valuable. The truth is, the human mind can’t navigate this extraordinary amount of multidimensional data in any way that could be considered effective or practical.
NTT Com: But we can build machines to analyze the data, and then make decisions based on the outputs.
Gupta: That’s how it continues. The machine will only take you so far. I learned this from the world of chess. Almost anything we do has been repeated in one form or another in some venue of life. This whole revolution has been played out in the world of chess. These advanced machines can now beat any human—a world champion, grandmaster, whoever. But what few people know is that an average chess player with a chess machine that they know how to use will beat any world champion or grandmaster on a consistent basis. It is this combination that we are trying to create through our solutions. In the Vektor platform, we always have transparency in mind. We get the data through the ETL layer, transform it with our SigGMS, and then publish it in a visualization platform, which lets people take the output of the machine and make it their own—stress testing it, doing scenario planning or applying their judgment to it. This amalgam creates the real value.
NTT Com: Talk a little more about the importance of this human judgment in the final decisions made with data.
Gupta: Whether you rely on the machine or you rely on your own judgment, you are making a choice. If you say that the machine is correct, you are basically saying that the machine can exercise a judgment better than you. Therefore, you let the machine rule. Let me give an example. When the financial meltdown occurred, all these machines stopped working. The machine intelligence was ultimately a function of historical information. And so the machine could not easily pick out factors that were outside of that historical information or structural changes in the environment. Yet a person with some reasonable understanding of the situation could tell from the environment that home prices were way too high or that there was some uncertainty. So we allow the user to ask questions like, “In this county, if house prices went up or down by X, what would happen to my decision?” The realm of the “what if?” is not what machines do. Machines can enable these decisions; they can predict something using evidence embedded in history. But humans often fill in the blanks by drawing on experience from other parts of their life. This algorithmic efficiency of the human brain is not in the purview of the machine. While the machine uses certain types of information far more effectively than a human could, it cannot deal with completely extraneous things that are not captured in history. But the human mind can.
NTT Com: It seems we’re just starting to get a sense of what we may be able to accomplish by using big data. What do you think may be possible once we really get the hang of using this resource?
Gupta: What we are looking at is a nexus point in industry, far greater than the last significant IT transformation. First, we had ERP in the late-80s. Then came the whole Internet revolution. I think big data is, in a sense, the third wave. This big data revolution is like saying the atmosphere now has more oxygen in it. There is no element of what people do that is not being fundamentally transformed. I was talking to the CEO of a very large company the other day, and he said, “The problem I have is that in two years, I’m going to 100x the data that I have today.” His infrastructure is simply not designed for that type of complicated environment. The rate of data generation is far greater than the ability to capture and mine the data. He wanted to know what his infrastructure should look like, how his business process should change, how to even think about this problem. When we work with these companies, we find there is no aspect of the company that is not fundamentally changing because of data’s effect on its productivity. One of our clients is going through a global process of examining every single business that they have to see how big data is transforming them and what business opportunities are emerging as a consequence. Another company in the healthcare arena, when we spoke, I discovered that the data in the company is worth more than the company itself. The company is a $6 billion hospital chain. I really cannot think of a single industry that will not face a substantial transformation.
