Increasing customer value by integrating
Data Mining and Campaign Management software

Direct Marketing Magazine, February 1999

by Kurt Thearling

[A companion slide presentation is also available for this article.]


As a database marketer, you understand that some customers present much greater profit potential than others. But, how will you find those high-potential customers in a database that contains hundreds of data items for each of millions of customers?

Data Mining software can help find the "high-profit" gems buried in mountains of information. However, merely identifying your best prospects is not enough to improve customer value. You must somehow fit your Data Mining results into the execution of marketing campaigns that enhance the profitability of customer relationships.

Unfortunately, Data Mining and Campaign Management technologies have followed separate paths – until now. Your organization stands to gain a competitive edge by understanding and utilizing this new union.

This white paper describes how you can profit from the integration of Data Mining and Campaign Management technologies.

 

Introduction

To be successful, database marketers must, first, identify market segments containing customers or prospects with high profit potential and, second, build and execute campaigns that favorably impact the behavior of these individuals.

The first task, identifying market segments, requires significant data about prospective customers and their buying behaviors. In theory, the more data the better. In practice, however, massive data stores often impede marketers, who struggle to sift through the minutiae to find the nuggets of valuable information.

Recently, marketers have added a new class of software to their targeting arsenal; Data Mining applications automate the process of searching the mountains of data to find patterns that are good predictors of purchasing behaviors.

After mining the data, marketers must feed the results into Campaign Management software that, as the name implies, manages the campaign directed at the defined market segments.

In the past, the link between Data Mining and Campaign Management software was mostly manual. In the worst cases, it involved "sneaker net," creating a physical file on tape or disk, which someone then carried to another computer, where they loaded it into the marketing database.

This separation of the Data Mining and Campaign Management software introduces considerable inefficiency and opens the door for human errors. Tightly integrating the two disciplines presents an opportunity for companies to gain competitive advantage.

Some Definitions

A data warehouse is a repository for relevant business data. While traditional databases primarily store current operational data, data warehouses consolidate data from multiple operational and external sources in order to attain an accurate, consolidated view of customers and the business.

Database Marketing uses information in computerized databases to target offerings to customers and prospects.

Data Mining uses technologies such as neural networks, decision trees or standard statistical techniques to search large volumes of data. In doing so, Data Mining builds models for patterns that accurately predict customer behavior.

Scoring uses a model to predict future behavior. The score assigned to each individual in a database indicates that person’s likelihood of exhibiting a particular customer behavior.

Campaign Management uses information in a data warehouse or marketing database to plan, manage and assess marketing campaigns designed to impact customer behavior.

A customer segment is a group of prospects or customers who are selected from a database based on characteristics they possess or exhibit.

Scoring on the fly or dynamic scoring is the ability to score an already-defined customer segment within a campaign-management tool. Rather than scoring an entire database, dynamic scoring works with only the required customer subsets, and only when needed.

Attrition, sometimes known as churn, occurs when a customer terminates his or her relationship with a service provider. Marketing efforts usually focus on minimizing churn because the cost of bringing a customer back is usually much greater than the cost of retaining the customer in the first place.

 

Data Mining defined

Data Mining, by its simplest definition, automates the detection of relevant patterns in a database. For example, a pattern might indicate that married males with children are twice as likely to drive a particular sports car than married males with no children. If you are a marketing manager for an auto manufacturer, this somewhat surprising pattern might be quite valuable.

However, Data Mining is not magic. For many years, statisticians have manually "mined" databases looking for statistically significant patterns.

Today, Data Mining uses well-established statistical and machine learning techniques to build models that predict customer behavior. The technology enhances the procedure by automating the mining process, integrating it with commercial data warehouses, and presenting it in a relevant way for business users.

The leading Data Mining products, such as those from companies like SAS and IBM, are now more than just modeling engines employing powerful algorithms. Instead, they address the broader business and technical issues, such as their integration into today’s complex information technology environments.

In the past, the hyperbole surrounding Data Mining suggested that it would eliminate the need for statistical analysts to build predictive models. However, the value that an analyst provides cannot be automated out of existence. Analysts will still be needed to assess model results and validate the reasonability of the model predictions. Since Data Mining software lacks the human experience and intuition to recognize the difference between a relevant and an irrelevant correlation, statistical analysts will remain in high demand.

 

The purpose of Data Mining

Data Mining helps marketing professionals improve their understanding of customer behavior. In turn, this better understanding allows them to target marketing campaigns more accurately and to align campaigns more closely with the needs, wants and attitudes of customers and prospects.

If the necessary information exists in a database, the Data Mining process can model virtually any customer activity. The key is to find patterns relevant to current business problems.

Typical questions that Data Mining answers include:

Answers to these questions can help retain customers and increase campaign response rates, which, in turn, increase buying, cross-selling and return on investment (ROI).

 

Scoring the model

Data Mining builds models by using inputs from a database to predict customer behavior. This behavior might be attrition at the end of a magazine subscription, cross-product purchasing, willingness to use an ATM card in place of a more expensive teller transaction, and so on.

The prediction provided by a model is usually called a score. A score (typically a numerical value) is assigned to each record in the database and indicates the likelihood that the customer whose record has been scored will exhibit a particular behavior.

For example, if a model predicts customer attrition, a high score indicates that a customer is likely to leave, while a low score indicates the opposite. After scoring a set of customers, these numerical values are used to select the most appropriate prospects for a targeted marketing campaign.

 

The role of Campaign Management software

Database marketing software enables companies to deliver to customers and prospects timely, pertinent, and coordinated messages and value propositions (offers or gifts perceived as valuable).

Today’s Campaign Management software goes considerably further. It manages and monitors customer communications across multiple touch-points, such as direct mail, telemarketing, customer service, point-of-sale, e-mail and the Web.

Campaign Management automates and integrates the planning, execution, assessment and refinement of possibly tens to hundreds of highly segmented campaigns running monthly, weekly, daily or intermittently. The software can also run campaigns that are triggered in response to customer behavior or milestones – such as the opening of a new account.

 

Increasing customer lifetime value

Consider, for example, customers of a bank who only use the institution for a checking account. An analysis reveals that after depositing large annual income bonuses, some customers wait for their funds to clear before moving the money quickly into their stock-brokerage or mutual fund accounts outside the bank. This represents a loss of business for the bank.

To persuade these customers to keep their money in the bank, marketing managers can use Campaign Management software to immediately identify large deposits and trigger a response. The system might automatically schedule a direct mail or telemarketing promotion as soon as a customer’s balance exceeds a predetermined amount. Based on the size of the deposit, the triggered promotion can then provide an appropriate incentive that encourages customers to invest their money in the bank’s other products.

Finally, by tracking responses and following rules for attributing customer behavior, the Campaign Management software can help measure the profitability and ROI of all ongoing campaigns.

 

Integrating Data Mining and Campaign Management

The closer Data Mining and Campaign Management work together, the better the business results. Today, Campaign Management software uses the scores generated by the Data Mining model to sharpen the focus of targeted customers or prospects, thereby increasing response rates and campaign effectiveness.

Unfortunately, the use of a model within Campaign Management today is often a manual, time-intensive process. When someone in marketing wants to run a campaign that uses model scores, he or she usually calls someone in the modeling group to get a file containing the database scores. With the file in hand, the marketer must then solicit the help of someone in the information technology group to merge the scores with the marketing database.

This disjointed process is fraught with problems:

The solution to these problems is the tight integration of Data Mining and Campaign Management technologies. Integration is crucial in two areas:

First, the Campaign Management software must share the definition of the defined campaign segment with the Data Mining application to avoid modeling the entire database. For example, a marketer may define a campaign segment of high-income males between the ages of 25 and 35 living in the northeast. Through the integration of the two applications, the Data Mining application can automatically restrict its analysis to database records containing just those characteristics.

Second, selected scores from the resulting predictive model must flow seamlessly into the campaign segment in order to form targets with the highest profit potential.

 

The integrated Data Mining and Campaign Management process

This section examines how to apply the integration of Data Mining and Campaign Management to benefit the organization. The first step creates a model using a Data Mining tool. The second step takes this model and puts it to use in the production environment of an automated database marketing campaign.

Step 1: Creating the model

An analyst or user with a background in modeling creates a predictive model using the Data Mining application. This modeling is usually completely separate from campaign creation. The complexity of the model creation typically depends on many factors, including database size, the number of variables known about each customer, the kind of Data Mining algorithms used and the modeler’s experience.

Interaction with the Campaign Management software begins when a model of sufficient quality has been found. At this point, the Data Mining user exports his or her model to a Campaign Management application, which can be as simple as dragging and dropping the data from one application to the other.

This process of exporting a model tells the Campaign Management software that the model exists and is available for later use.

Step 2: Dynamically scoring the data

Dynamic scoring allows you to score an already-defined customer segment within your Campaign Management tool rather than in the Data Mining tool. Dynamic scoring both avoids mundane, repetitive manual chores and eliminates the need to score an entire database. Instead, dynamic scoring marks only relevant customer subsets and only when needed.

Scoring only the relevant customer subset and eliminating the manual process shrinks cycle times. Scoring data only when needed assures "fresh," up-to-date results.

Once a model is in the Campaign Management system, a user (usually someone other than the person who created the model) can start to build marketing campaigns using the predictive models. Models are invoked by the Campaign Management System.

When a marketing campaign invokes a specific predictive model to perform dynamic scoring, the output is usually stored as a temporary score table. When the score table is available in the data warehouse, the Data Mining engine notifies the Campaign Management system and the marketing campaign execution continues.

 

Here is how a dynamically scored customer segment might be defined:

Where

Length_of_service = 9

and

Average_balance > 150

and

In_Model(promo9).score > 0.80


In this example:

  • Length of service =9 limits the application of the model to those customers in the ninth month of their 12-month contracts, thus targeting customers only at the most vulnerable time. (In reality, there is likely a variety of contract lengths to consider this when formulating the selection criteria.)
  • Average balance > 150 selects only customers spending, on average, more than $150 each month. The marketer deemed that it would unprofitable to send the offer to less valuable customers.
  • Promo9 is the name of a logged predictive model that was created with a Data Mining application. This criterion includes a threshold score, 0.80, which a customer must surpass to be considered "in the model." This third criteria limits the campaign to just those customers in the model, i.e. those customers most likely to require an inducement to prevent them switching to a competitor.

 

Data Mining and Campaign Management in the real world

Ideally, marketers who build campaigns should be able to apply any model logged in the Campaign Management system to a defined target segment. For example, a marketing manager at a cellular telephone company might be interested in high-value customers likely to switch to another carrier. This segment might be defined as customers who are nine months into a twelve-month contract, and whose average monthly balance is more than $150.

The easiest approach to retain these customers is to offer all of them a new high-tech telephone. However, this is expensive and wasteful since many customers would remain loyal without any incentive.

Instead, to reduce costs and improve results, the marketer could use a predictive model to select only those valuable customers who would likely defect to a competitor unless they receive the offer.

 

Evaluating the Benefits of a Data Mining Model

 

The chart to the left, called a "gains chart," suggests some benefits available through Data Mining. The diagonal line illustrates the number of responses expected from a randomly selected target audience. Under this scenario, the number of responses grows linearly with the target size.

The top curve represents the expected response if you allows the model scores to determine the target audience. The target is now likely to include more positive responders than in a random selection of the same size. The shaded area between the curve and the line indicates the quality of the model. The steeper the curve, the better the model.

Other representations of the model often incorporate expected costs and expected revenues to provide the most important measure of model quality: profitability. A profitability graph like the one shown below can help determine how many prospects to include in a campaign. In this example, it is easy to see that contacting all customers will result in a net loss. However, selecting a threshold score of approximately 0.8 will maximize profitability.

For a closer look at how the use of model scores can improve profitability, consider an example campaign with the following assumptions:

As the table below shows, a random sampling of the full customer/prospect database produces a loss regardless of the campaign target size. However, by targeting customer using a Data Mining model, the marketer can select a smaller target that includes a higher percentage of good prospects. This more focused approach generates a profit until the target becomes too large and includes too many poor prospects.

 

Campaign Size

Cost

Random Selection

Targeted Selection

Response

Revenue

Net

Response

Revenue

Net

100,000

$100,000

2,000

$80,000

($20,000)

4,000

$160,000

$60,000

400,000

$400,000

8,000

$320,000

($80,000)

30,000

$1,200,000

$800,000

1,000,000

$1,000,000

20,000

$800,000

($200,000)

35,000

$1,400,000

$400,000

2,000,000

$2,000,000

40,000

$1,600,000

($400,000)

40,000

$1,600,000

($400,000)

 

Conclusion: The Benefits of integrating Data Mining and Campaign Management

For marketers:

Records can be scored when campaigns are ready to run, allowing the use of the most recent data. "Fresh" data and the selection of "high" scores within defined market segments improve direct marketing results.

Scoring takes place only for records defined by the customer segment, eliminating the need to score an entire database. This is important to keep pace with continuously running marketing campaigns with tight cycle times.

Accelerated marketing "velocity" also increases the number of opportunities used to refine and improve campaigns. The end of each campaign cycle presents another chance to assess results and improve future campaigns.

For statisticians:




[ Data Mining Page ] [ White Papers ] [ Data Mining Tutorial ]