Great Case Study on KeyBank

Here is a  terrific article (KeyBank Case Study) on how KeyBank (KEY) is increasing profitability while improving customer satisfaction thought the use of predictive models, behavioral event scoring and targeted strategies. Chip Clarke, senior vice president of strategic analytics, talks about both developing targeted segments based on structured and unstructured data and driving strategies and specific treatments for each segment.

To read the whole article see: KeyBank Case Study at 1to1 Magazine

Almost caught up to where we were in 2001

It's funny that for the most part Internet retailers and the rewards card issuers are just now catching up to where the vendors of Internet personalization, targeting, data-mining, etc... wanted them to be in 2001. For example, a friend of mine used his rewards card to purchase a router at a local Staples store. A couple days later he received this personalized email.

Now I'm not saying that others haven't been doing this, in fact I know of several companies with fairly advanced campaign targeting systems in place. What I like is that this email is essentially trigger based and personalized and from a standard retailer.

Text Mining for Predictive Modeling

Last week a colleague of mine involved in a product evaluation of text mining tools for predictive modeling began asking questions about the "proper" criteria for selection and about the necessary features for a successful product. As the conversation turned into a request for a write up I remembered an article that I wrote a while ago entitled The Next Wave in Customer Analytics for DM Review where I talked about what it takes to use text effectively for production quality predictive models.

A/B Testing with PREDIGY

The need for a great platform to plan, perform, manage, track and report on A/B tests is of critical importance to our customers as they compete to make better, more profitable data-driven decisions. Often referred to as champion-challenger tests in the banking and risk management worlds, the steps, systems and best practices to make this work can be daunting. At Intelligent Results we've done our best to unite these in single platform, PREDIGY. I've listed several of the key features that PREDIGY provides that you really should know about. This is certainly not an exhaustive list but these little details make being data driven much, much easier.

1. IR_ControlNumber - PREDIGY creates and maintains a variable called "IR_ControlNumber" for every dataset loaded into it. This random variable is created at both design-time and run-time allowing it to be used as the basis for strategy splits. IR_ControlNumber is a single numeric variable between 0 and 999 randomly assigned to every account. While each account's number is completely random it can be consistently generated for the same record meaning that the same record will always get the same IR_ControlNumber. This makes this variable ideal for champion-challenger tests over a single campaign or for extended duration tests.
   




2. Segment Name and Strategy Code - Each end node or leaf of the strategy tree has both a Segment Name and a Strategy Code.
   
   a. The Segment Name is a unique identifier for that group of accounts. PREDIGY enforces that these Segment Names are unique which means that at scoring time each account will fall into one and only one segment and that the Segment Name for each account will be logged to the IR Report database. Examples below include "Low Value Control Segment" and "High Value Dialing Segment".
   
   b. The Strategy Code is a non-unique code or score that the strategy will assign to each record at scoring time. You can see below that two of the segments both use "BAU" as the Strategy Code. This means that downstream systems that take action off of this "BAU" code will not know whether the account came from the high or low value segment. Both the Segment Name and Strategy Code for a given account are always available for reporting out of IR Report.

3. IR Report – IR Report is designed to capture and report on operational PREDIGY scoring and decisioning applications. Basically, the way it works is that whenever accounts are scored on the IR Production Engine a series of files are written which include all of the input and output values for every record processed. This data is automatically loaded into the IR Report database. A loading program called IRReportDBLoader.exe manages this process, including schema generation, loading and marking files that have already been loaded. Reports can then be run against this IR Report database using PREDIGY’s embedded Crystal Reports or any other query and reporting tools. In general 3 types of reports can be created from IR Report: IT reports focused on the things IT cares about like processing speed, errors, etc...; Statistical reports monitoring input and output variable and score distributions to guard against drift and untimely model aging; and finally, Strategy reports allowing for A/B testing and measurement of the effectiveness of strategies. For these Strategy reports to be complete it is necessary to append outcome data to the database once that data becomes available. What will already be logged into IR Report will include which Segment Name (unique leaf node) and Strategy Code (action that operations should have taken, potentially common to multiple segments) each account fell into on every scoring run. It’s also important to remember that any other account level information provided or generated at scoring time can also be logged into the IR Report database; such as the accounts score on one or more models or strategies (regardless of whether or not they a re used in the current strategy), balance, state, etc...
   




4. Formulas and Actions – The formulas and actions created in design time provide users a simple yet powerful way to estimate and simulate how accounts will flow through their run-time application and what that will mean to their business. In the example below we have calculated several simple measures based on the historic performance of the accounts already loaded into PREDIGY. These include the number and percentage of accounts, the percentage of good and bad accounts and the sum of payments from those accounts in a 30 day period. These formulas and any others that the user wants are easily applied to each node in the decision tree and are updated instantly as the user changes their business rules, models, split points, etc... The Actions and their included formulas combine both calculated measures from the existing records with additional information supplied by the business user allowing you to create estimates, simulations and business cases directly in-line with your decision tree. For example, when we calculate the cost of the "ACTION: Demand Letter" (actions are circled in purple) the formula includes a user defined constant for each letter that will have to be sent for each account. Creating Formulas and Actions is very simple and both can be used throughout the decision tree for data analysis and predicting future business results. Formulas and Actions are purely informational and do not have any impact on the operational scoring process regardless of which actions or formulas you put on various end-nodes or leaves.To affect a downstream system's process you should use Strategy Codes as mentioned above.

I'm sure there are things I've left out of this write-up but it should give you a starting point to investigate using PREDIGY to manage a larger part of the Modeling, Decisioning, Scoring, Tracking, Reporting and Improving life-cycle. As you read this please feel free to comment and because I've skimmed over several sections like deploying and configuring your application (IRX file), the IR Production Engine and it's web-services and batch options, IR Report, etc... please don't hesitate to send me an email.

Triggers in a data driven world

To begin talking about and understand triggers in a data-driven world one needs to think about an entire strategy including models, data variables, business rules and segment based outcomes as a trigger. In fact, what makes it a trigger is more how it is used then the type of data that it processes. A trigger can and often should contain all the complexity (models, rules, data transformations, etc...) as full blown campaigns. The key is that triggers are executed frequently: whenever data changes and/or there is an opportunity to take the right data-driven action. For a company like Intelligent Results whose production engine can easily be called in real-time or batch this isn't a big deal; however, for many organizations that use scores/models and decision trees today this is difficult because they have to recode those models and decision rules into other operational systems to make them work. So once you think of the entire decision tree with all its scores, variables and actions as a trigger you can imagine that any type or types of data and analysis could go into the trigger. For example, a trigger might in fact be statistically based, like a score with cut points, or might be based on the occurrence of a words or concept in a call center conversation. What's more a trigger can also involve multiple elements like a cross-sell offer trigger which initiates an offer if the transaction value of the current shopping cart is >25% over the 6 month average or the predictive model score, p(acceptance), for the optimal product is 35%, or the customer record in the past contains the concept of "moving or relocation." In that example, text, temporal roll-ups, transformations (feature math) and predictive models would all have to interact to fire a "trigger" and initiate the Action. Triggers are an awesome concept for Intelligent Results and we should never miss an opportunity to bring them into the conversation. They are hot in the industry right now and they rely on several concepts that we've long supported:

• Execute Often - and we really mean score, evaluate and decision often. Triggers are often used in real-time or fast batches and we believe this is important because you want to be able to react as soon as the information changes or as soon as the customer gives you an opening.




• Use Scores and Decisions - Triggers are more than scores and data. Triggers may require scores and data which we provide, but they also require decisions, cut-points, thresholds and actions. Only PREDIGY gives customers the ability to do all that in one system.




• Use All Your Data - Powerful Triggers can be based on all types of data, used independently and in combination. From unstructured text and sequence data to data comparisons (transformation in feature math) and statistical models PREDIGY allows users to use and combine all types of data into simple and complex business rules and triggers.




• Rapid Production - Triggers are more valuable when they can be quickly designed, simulated, implemented and executed. PREDIGY makes all that fast and reliable. The PREDIGY application factory concept integrates and streamlines each of these steps. And of course with the IR Production Engine your Triggers can be easily integrated, deployed and distributed across multiple operational systems and client interaction channels.




• Track Everything - the PREDIGY platform does that.

Below I've added 2 examples of Triggers that a banking call center customer might use.

The first, is an example of a skills based routing trigger used by a credit card company to route inbound emails about AOL Billing issues to specialized agents. Without going into to much detail on why this is necessary (that's a whole other posting on discovering attrition drivers) you can see that if the email is about AOL charges then the customer is to be routed to a special queue. This is a very simple text based trigger that could be executed by the IR Production Engine against emails, call center call, web forums, etc... The IR Production Engine makes it possible for a bank to run this strategy in real-time or in batch.

The second example is a bit more complex. It's designed to allow call center agents to constantly, profitably and quickly respond to customer's rate adjustment requests or to proactively offer rate adjustments to particularly valuable segments. The first decision element looks at a customer's current interest rate and compares it to the rate they could get from industry competitors. If the delta is within 2 points no action is taken. If the delta is 2 points or greater then the value of the customer is assessed. If they are not a high spender then the account must be sent for additional review (in fact the entire rate management strategy behind this additional review could also be run in real-time if the bank wanted to but that's a different posting). If the customer is a high spender then the call center agent is empowered to make an immediate rate adjustment. This simple, real-time data-driven strategy improves customer satisfaction, regulatory compliance and profitability.

Intelligent Results Website Updated

The new website looks great! Intelligent Results