Context Matters When Modeling Human Judgment and Choice

Herbert Simon was succinct when he argued that judgment and choice "is shaped by a scissor whose two blades are the structure of the task environment and the computational capabilities of the actor" (Simon, 1990, p.7). As a marketing researcher, I take Simon seriously and will not write a survey item without specifying the respondent's task and what cognitive processes will be involved in the task resolution.

Thus, when a client asks for an estimate of the proportion of European car rentals made in the United States that will be requests for automatic transmissions, I do not ask "On a scale for 1=poor to 10=excellent, how would you rate your ability to drive a car with a manual transmission?" Estimating one's ability, which involves an implicit comparison with others, does not come close to mimicking the car rental task structure. Nor would I ask for the likelihood of ordering an automatic transmission because probability estimation is not choice. Likelihood will tend to be more sensitive to factors that will never be considered when the task is choice. In addition, I need a context and a price. It probably makes a difference if the rental is for business or personal use, for driving in the mountains or in city traffic, the size of the vehicle, and much more. Lastly, the proportion of drivers capable of using a stick shift increases along with the additional cost for an automatic transmission. Given a large enough incremental price for automatic transmissions, many of us will discover our hidden abilities to shift manually.

The task structure and the cognitive processing necessary to complete the task determine what data need to be collected. In marketing research, the task is often the making of a purchase, that is, the selection of a single option from many available alternatives. Response substitution is not allowed. A ranking or a rating alters the task structure so that we are now measuring something other than what type of transmission will be requested. Different features become relevant when we choose, when we rate, and when we rank the same product configurations. Moreover, the divergence between choice and rating is only increased by repeated measures. The respondent will select the same alternative when minor features are varied, but that respondent will feel compelled to make minor adjustments in their ratings under the same conditions. Task structure elicits different cognitive processing as the respondent solves different problems. Ratings, ranking and choice are three different tasks. Each measures preference as constructed in order to answer the specific question.

Context matters when the goal is generalization, and one cannot generalize from the survey to the marketplace unless the essential task structure has been maintained. For example, I might wish to determine not only what type of car you intend to rent in your next purchase but what you might do over your next ten rentals. Now, we have changed the task structure because car rentals take place over time. We do not reserve our next ten rentals on a single occasion, nor can we anticipate how circumstances will change over time. The "next ten purchases question" seems to be more a measure of intensity than anticipated marketplace behavior.

Nor can one present a subset of available alternatives and ask for the most and least preferred from this reduced list without modifying the task structure and the cognitive processing used to solve the tasks. The alternatives that are available frame the choice task. I prefer A to B until you show me C, and then I decide not to buy anything. Or, adding a more expensive alternative to an existing product configuration increases the purchase of medium priced options by making them seem less expensive. Context matters. When we ignore it, we lose the ability to generalize our research to the marketplace. Finally, self-reports of the importance or contribution of features are not context-free; they simply lack any explicit context so that respondents can supply whatever context comes to mind or they can just chit-chat.

The implications for statistical modeling in R are clear. We begin with a description of the marketplace task. This determines our data collection procedures and places some difficult demands on the statistical model. For example, purchase requires a categorical dependent variable and a considerable amount of data to yield individual estimates. Yet, we cannot simply increase the number of choice sets given to each respondent because repeated measures from the same individual alters that individual's preferences (e.g., price sensitivity tends to increase over repeated exposures to price variation). Bayesian modeling within R allows us to exploit the hierarchical structure within a data set so that we can use data from all the respondents to compensate for our inability to collect much information from any one person. However, borrowing data from others in hierarchical Bayes is not unlike borrowing clothes from others; the sharing works only when the others are exchangeable and come from the same segment with the a common distribution of estimates.

None of this seems to be traditional preference elicitation, where we assume that preference is established and well-formed, requiring only some means for expression. Preference or value is the latent variable responsible for all observed indicators. Different elicitation methods may introduce some unique measurement effects, but they all tap the same latent construct. Simon, on the other hand, sees judgment and decision making as a form of problem solving. Preferences can still be measured, but preferences are constructed as solutions to specific problems within specific task structures. Although preference elicitation is clearly not dead, we can expect to see increasing movement toward context awareness in both computing and marketing.