Application of Dynamic Programming Method to Marketing Decisions Based on Customer Database

1 Introduction

Recently, a lot of researchers pay more attention to the customer purchase behavior based on the different marketing stimulis which include the short term promotions and long term programs[1–3]. The short term promotions are usually in a form of the coupon, different shipping policy and instant discount. The long-term programs, which are often called customer reward programs, are designed to maintain and enhance customer loyalty. As one of most popular marketing strategies, customer reward programs have attracted considerable interest among researchers[4–7]. The goal of reward programs is to encourage customer frequent purchase behavior which is potentially beneficial to the firm in a form of points, frequent flyer mileage, free gifts, and so on[3]. In practice, a loyalty card, rewards card, point card, advantage card, or club card is a plastic or paper card, visually similar to the credit card or debit card, which can identify the card holder as a member in a customer reward program[3]. In addition, the programs also can be viewed as a dynamic incentive based on the cumulative purchase amount in a certain period, which shifts the customer purchase behavior from myopic or single period decision making to a dynamic or multiple period decision making[8].

What this research focus is how the customer reward programs stimulate the customer purchase behavior. We adopt the dynamic programming (DP) algorithm to analyze the influence of such reward programs, since the customer makes decisions not only based on the current rewards from the purchase but also the future rewards received from the customer reward programs[3]. The approach is based on the idea that a customer’s observed sequence of decisions may be interpreted as the solution to a dynamic optimization problem. DP method is ideal for analyzing individual choices which are based on both current and future expected benefits[8]. That a reward program rewards customer on the level of purchasing over a specified period is a prime example of such a decision problem. Another advantage of dynamic programming method is that the estimated coefficients can be used to conduct simulations which replicate the consumer’s dynamic decision process[9].

2 Theoretical Background

3 The dynamic Formulation Procedure and Algorithm

3.1 Dynamic Customer Behavior in a Reward Program

This part develops a model of customer response to a reward program. In a reward program, customer purchase behavior in each period of whole year can be formulated based on their current cumulative purchase. The whole year is divided into 52 weeks according to the real happened purchase data. At the end of the year, the merchant will distribute the reward on the basis of customer whole year cumulative purchase amount. However, after this time period, the total purchase amount will restart from zero. It means that even the customer’s cumulative purchase amount is very big, but still could lose the chance to get the reward just because of not touching the reward level that the merchant sets. Therefore the reward programs will dynamically stimulate the customer’s purchase behavior based on the reward level and reward scale they set. According to the date, customer has 4 choices in each period, that is, no purchase, small purchase, medium purchase and large purchase respectively. The price in each period is supposed to randomly change associated with the last period price and the purchase level the customer chose in last period. Due to lack of real price data, it is assumed that the price for each period is uniform distribution and has three different levels which are high price, medium price and low price. As a result, the customer state can be defined in each period as two dimensions, which is cumulative purchase amount and price separately.

Because of the use of the part of the paper frame, we define the customer state and stochastic factor from the paper. However, due to the inaccessible of the data and all the parameter that the author estimated, we cannot consider all the factors influencing the customer purchase behavior, the probability evolution of the price for next period and write down the clear expression of the reward function under different actions. However, we give the reasonable estimate of the parameter inside the reward function by using two piece of common knowledge. Firstly, the marginal reward is decreasing as the increase of the cumulative purchase. Secondly, to the same cumulative purchase amount, the higher the current price is, the less reward the customer gets. With the known data, we assume that the customer purchase is a multi-periods decision making and could be replicated by a DP method, then estimate the parameters from the data and DP algorithm to see the influence of the short-term promotion and reward programs. Whereas we use known parameter from Lewis[3] to see the customer’s each period action by adopting the DP algorithm.

3.2 Formulation as a DP Algorithm

The state space is a vector of information about the environment that is relevant to the customer’s forward-looking optimization problem. The state space may consist of marketing-mix elements, such as the pricing environment in a given week, and customer-specific information, such as cumulative purchase amount. State x(t) denotes the cumulative purchase amount at period t. p(t) denotes the price of the merchandise at period t where p(t) includes the high price and mediate price and low price. The quantity decision is likely to be based on marketing factors such as price and on individual-level factors such as inventories. A myopic decision maker would select from the options 1, 2, ⋯, J. Actions are as follows:

j = 1 no purchase;

j = 2 purchase small [0, 50);

j = 3 purchase medium [50,75);

j = 4 purchase large [75, +∞).

The increase of cumulative purchase amount is a comparative number on average. W is a stochastic factor, which denoted as w1: price high; w2: price medium; w3: price low. We assume the next period price could be high, mediate, low with uniform distribution.

The evolution of the state is

[Xt+1,Pt+1]=[Xt+∑j=14[33(j−1)dj(t),w(t)]],

where ∑j=14dj(t)=1. It means that customer only can choose one action in each period.

The reward functions for different action in each period are as follows:

Rno(t)=bno;Rsm(t)=bsm−bp,smPt−[(x(t)+33(jsm−1))/100]bsmde;Rmed(t)=bmed−bp,medPt−[(x(t)+33(jmed−1))/100]bmedde;Rlrg(t)=blrg−bp,lrgPt−[(x(t)+33(jlrg−1))/100]blrgde.

We set three level reward trigger in last period. Therefore, the reward for the last period T are as follows:

Rno(t)=bno;Rsm(T)=bsm+bp,smPT+bc,smCT+∑h=1Hbh,smSH(h)T+(∑level=13blevel)(XT−1+33);Rmed(T)=bmed+bp,medPT+bc,medCT+∑h=1Hbh,medSH(h)T+(∑level=13blevel)(XT−1+66);Rlrg(T)=blrg+bp,lrgPT+bc,lrgCT+∑h=1Hbh,lrgSH(h)T+(∑level=13blevel)(XT−1+j).

Where we have

IfXT−1+33(j−1)<1000,thenblevel=1=blevel=2=blevel=3=0;If1000≤XT−1+33(j−1)<2000,thenblevel=1=0.1,blevel=2=blevel=3=0;If2000≤XT−1+33(j−1)<3000,thenblevel=1=0,blevel=2=0.5,blevel=3=0;If3000<XT−1+33(j−1),thenblevel=1=blevel=2=0,blevel=3=0.8.

The Bellman equation is

J(t)=maxj∈J∗{Rj(t)+αEpw⁡Jj(t+1)},

where α is discount factor. When α is equal to 0, it means that the customer doesn’t care the next period reward. A finite period DP algorithm is used in this research. According to the customer purchase habit, the time horizon is usually divided into 52 weeks. Here, to see the influence of reward programs and decrease of the calculation burden, we only consider 10 periods to the end of the year.

3.3 Results Analysis

We plot the policies the customer chose at one period ahead of the T for both with reward program and without reward program.

Figure 1

The policies chosen by the customer under the reward program

Here X axial denotes the cumulative purchase amount and the different price under the same cumulative purchase amount, whereas the Y axial denotes the different level of purchase the customer choose. 4 means buy large and 1 means buy nothing.

Figure 2

The policies chosen by the customer without reward program

We could see from these two figures that there is a jump in the first one. The place of the jump is just around the cumulative purchase approach to the 3000. In this cumulative purchase area which is below 3000 but approach 3000, the customer is willing to buy large to cross the reward level 3000 to get the reward in the final period. However, in the long term, the stimulation of reward program will move forward, like Figure 3.

Figure 3

Ten periods to the T with reward in T

Figure 4

Ten periods to T without reward in T

In a long term, the reward program influences all the customer purchase actions around the reward trigger level by the cumulative effect.

4 Conclusions

This article presents a DP method to modeling longitudinal customer purchase behavior. The model measures the influence of reward programs by considering customers’ sequences of purchase as a solution to a dynamic optimization problem. A primary strength of the approach is that the dynamic programming method provides a good solution for modeling dynamic customer behavior. The research shows that this model is a great tool by which to monitor or track the development of customer purchase behavior. The method includes both the influence of previous behavior in terms of cumulative purchase amount and expectations of future prices and loyalty rewards. Modeling the forward-looking behaviors intuitively appeal the customer response to a reward program[18]. With the popularity of reward programs, this paper provides a modeling method for evaluating the effectiveness of marketing strategy.

The model estimates the effects of the reward program. The results show that the reward program increases customer repeat-purchase rate effectively. Specially, the customers tend to increase their purchase to cross the reward level to get the reward in the final point of a reward period. However, in the long term, the stimulation of reward program will move forward. In a word, the reward program increases the customer’s purchase level actually and stimulates customer repeat purchase behavior. On the other hand, the model also provides a platform for conducting simulation studies because the estimated coefficients can be used to conduct simulations that replicate the consumer’s dynamic decision process.

5 Limitations and Future Research

The model is developed using data from internal transaction databases. When modeling customer behavior using this type of data, it is usually necessary to make reasonable assumptions about the customer purchase decision. In general, the DP method adopted in this research is to assume the presence of a relatively constant outside alternative. In both the newspaper subscription and Internet grocery purchase decisions the customer has a no buy option in each period. In general, neither dataset includes data or covariates that indicate how the relative attractiveness of the no buy or outside alternative change in each period. By way of compensation for this deficiency, prices are adjusted using relevant CPI data for comparable products. More specific variations in the outside alternatives are captured by the stochastic or error component.

In terms of future research this research focuses on what is hoped to be a series of studies focusing on the dynamics of customer behavior and customer management. The current work focuses mostly on the development of methodologies for customer analysis and management. The next step is to begin to examine both databases in an effort to test and add to the extant body of knowledge related to customer loyalty.