What do a YouTube video and operational intelligence have in common?

In 2011, DoSomething.org, a non-profit organization, posted a video on YouTube featuring celebrities asking young people to donate used sports equipment to youth in need. That video was viewed by 1.5 million people. Success, you might think? Well, only eight viewers signed up to donate. Of those eight, none actually donated.

What happened? How could such a huge number of views not produce any donations? “We were concerned with the wrong metric,” writes Jeff Bladt, Director of Data Products & Analytics at DoSomething.org, in his blog. “In the case above, success meant donations, not video views. As we learned, there is a difference between numbers and numbers that matter. This is what separates data from metrics.”

Track metrics, not data. Tracking the wrong data left DoSomething.org with no donations. Likewise, a bank’s operational intelligence (OI) solution collecting the wrong data can produce ineffective and, worse, potentially misleading actionable information. There are various ways to pick the correct metrics for a bank’s OI purposes. Let’s look at one method.

Look at an individual’s performance objectives. The process of defining meaningful OI metrics starts with an individual. Each OI dashboard has been designed for one person (or a category of persons). That individual is expected to achieve measurable operational performance objectives. Let’s say that the individual this dashboard has been designed for is named Edward. He is the supervisor of a back-office operation for high-value payments. Edward’s operational objectives are to never miss a cut-off, always meet the Fed’s deadlines, and achieve higher satisfaction levels on customer surveys. These objectives are mapped to the bank’s customer commitments.

Map the person to the bank’s commitments, obligations, and deadlines. In our example, Edward’s objectives map to the bank’s:

  • Customer commitments (e.g., process all payments by cut-off)
  • Regulatory compliance obligations (e.g., meet the Fed’s deadlines)
  • Current strategic initiatives (e.g., improve customer experience) 

Once that has been mapped, the OI metrics need to align with these objectives.

Align the metrics. Select the metrics that align with Edward’s objectives and the bank’s goals. Let’s take the example of Edward’s objective to never miss a cut-off. The goal is to proactively identify and resolve process issues prior to putting such a cut-off at risk. Therefore the OI measurements and metrics must create the necessary situational awareness to alert Edward when a situation requires his immediate attention.

An example might be the measurement of abnormalities that could lead to putting a payments cut-off at risk. In this case, OI would measure payments stuck in a step for too long. And it would measure when an abnormally high number of payments fail their STP route and go into a manual repair queue. Other things that would be measured are files or messages not received on time from a customer. Any of those situations will alert Edward and trigger him to take proactive actions leading to the avoidance of missing the cut-off. In addition to the operational objectives, there’s one more measurement to take.

Measure the OI solution to ensure that the metrics matter. Do this by measuring the data before and after the OI solution has been in production. In our example of Edward, there could be two metrics for measuring the “before and after” data. One metric could be the number of payments that missed their cut-off as a percentage of the total payments processed daily. The second metric would be the value (i.e., the dollar amount) of missed payments as a percentage of the total value processed daily.

At first glance, getting 1.5 million YouTube views seemed like success for an organization. But it was a failure because their goal was donations, not views. Likewise, it’s important for a bank to look at its goals, and to map its OI solution to track the results that will achieve those goals. Remember:

  1. Look at the operational objectives of the person the OI dashboard was designed for
  2. Map those objectives to the bank’s commitments, obligations, and deadlines
  3. Align the metrics 

When you go through those three steps, you’ll find the measurements and metrics that matter.

Source: DoSomething.org blog post Know the Difference Between Your Data and Your Metrics

What larger banks do to minimize the difficulty of automating stress testing

This is an excerpt from a transcript of The Axway Podcast, “What larger banks do to minimize the difficulty of automating stress testing.”

ANNOUNCER: Forbes writer Tom Groenfeldt recently published a piece titled, “Compliance Efforts Can Bring Business Benefits for Banks,” and he quoted a risk consultant in it who noted that “For the larger banks, over $50 billion, it is more difficult to automate the stress testing because of the sheer volume of data and the number of systems they have in place. Big banks this year identified data integration as their big challenge. That was not the top challenge for banks with $10 to $50 billion.”

PETER BENESH: A lot of the challenges around data integration are the ability to access data that especially resides on mainframes and legacy systems. Or in this case, it may be in the cloud. Or it may not even be in a database. It could be in Excel spreadsheets. It could be unstructured data that doesn’t even reside in a formatted database.

ANNOUNCER: That’s Peter Benesh, Axway’s director of solution marketing for the Financial Services industry. We asked him to describe what those larger banks do to minimize the difficulty of automating stress testing.

PETER BENESH:  The bigger the bank is, the more likely it is that it has a greater variety of both hardware and software technologies. Probably the bigger they are, probably the more mainframes they might have. The more in-house developed reporting systems they may have. The data integration challenge really becomes “Does the technology that they have provide all of the various connectors that are required to enable their data integration platform to extract information from every possible data source?” The next challenge in that is once you extract it, depending on what the source is, the data formats are most likely going to be in various structures. In order to get all of that information into an analytic server, you’ve got to translate all of that into one common data structure. The greater variety of sources you have — not only do you need to have more connectors, but you also have to have a greater library of data transformation capabilities. Such that you can translate or transform data formats that are coming from Oracle, that are coming from a mainframe, that are coming from flat files. Because all of that has to be put into a common format that an analytic server like Hadoop, for example, can digest or ingest for analytics. That whole process is traditionally called ETL — extract, transform, load. That’s what that acronym means. We aren’t really in the business of ETL, per se, but to the extent that any of this data that they need to integrate also needs to come from outside their organization… Let’s say they have information in the cloud, or maybe they have information for partners that they want to integrate into these exercises. Anything that they would need to do more of a B2B-type integration with, bringing information through their firewalls… And, of course, our gateway technologies could help them with that. It’s really a function of how broadly do they want to scope the integration exercise. Do they want to restrict it simply to information that resides in their internal systems? If that’s the case, then pretty much a very robust ETL tool is what they need. If they want to expand the scope to include both internal and external data, then they need strong ETL and they need strong MFT.

To read the article in its entirety on Forbes.com, please click here.

To listen to the podcast on YouTube (audio only), please click here.