DOI: 10.31038/ASMHS.2019343

Abstract

The paper uses the emerging science of Mind Genomics to understand emotional responses (frustration) and prevention of decisions experienced when the respondent reads test vignettes describing websites which provide medical information (health) and/or medical insurance information (health-related finances). Respondents read and evaluated combinations of 2–4 messages (answers to questions), with the messages combined according to an experimental design. The ratings on a five- point scale provided an assessment of both estimated ‘frustration’ and estimated ‘difficulty to make a decision.’ The analysis related the presence/absence of the messages to both frustration and to inability to make a decision. Three mind-sets emerged, suggesting that the estimated frustration encountered in difficult web searches for healthcare information is not unidimensional. The three emergent mind-sets are: MS1 (moderate latent frustration), MS2 (little latent frustration but easily & strongly frustrated) and MS3 (a great deal of latent frustration, doing best with a very simple, direct search process). The paper concludes with the presentation of the PVI, personal viewpoint identifier, which allows the healthcare provider to understand the sensitivities of the prospect, in terms of what problems increase frustration for that prospect. The objective of the PVI is to improve the user experience by understanding the mind of the user.

Introduction

The use of the Internet for searching and finding health information is rising and is accompanied by the realization that the ‘experience’ itself must be made easier. We are no longer in the birthing years of the 1970’s – 1990’s, when simply having access to a large world of information sufficed, astonishing those who had grown up in a world where information was to be sought after, no matter what the difficulty [1]. Experts have been replaced by websites, by chat advisors, by guided searches, so much so that often there is no expert but rather guidance embedded in the software and the instructions emerging from the software. People often first search the internet for information about diseases, then talk to their friends, and then encounter the doctors [2]. For diseases such as cancer, in earlier days a death sentence for many has spawned an entire network of communications and information [3–5]. The same goes for diabetes [6] and for heart disease [7]. As a consequence, medical information, may be getting increasingly dense over time as medicine advances and the literature and alternative options become overwhelming, for example the “BELONG” community of cancer patients [8]. Much of this this transition and new world is contained within the words ‘user experience,’ a phrase which encompasses the range from one’s impression of the website to one’s experience with the website to achieve certain goals. In the previous generations of science this area would have been subsumed under the rubric ‘man-machine interaction’ in the world of ‘human factors.’

This paper focuses specifically on one aspect of the user experience, the source for diagnosis and treatment information, both in terms of medical information and in terms of medical coverage information. The objective is to quantify the important of different aspects of the search as they drive expected frustration and expected inability to make a decision [9]. A traditional strategy to obtain the information is by a guided interview. The research instructs respondents to answer questions about needs, asks about sources of frustration, and experienced challenges in choosing an answer. The study would also measure responses when participants are exposed to the actual information and instructed to make a decision. Our approach complements this typical study just described. Our experiment presents respondents with vignettes defining the situation and instructs the respondents to select the likely outcome based upon the description of the experience. The analysis deconstructs the response to these vignettes into the contribution of the different elements of the vignette as drivers of expected frustration.

Mind Genomics as an emerging science traces its history to a combination of statistics (experimental design [10] conjoint measurement as ways to study decision-making [11, 12]. Mind Genomics expands topics from the laboratory out to everyday life. Furthermore, Mind Genomics expands the capabilities of design of experiments, using individual permutations of a basic, fundamental design. The consequence is that one need not overthink the selection of combinations of elements to go into the design. The permutation covers a wider amount of the space, analogous to the way the MRi in medicine takes many pictures of underlying tissue, not just the ‘correct one’, which may not even be known [13,14] The result has been the creation of a new science with applications from policy to products, from law to health and everyday life [15,16].

Mind Genomics method

Mind Genomics approaches the problem by an easy to construct, easy to analyze experiment. The experiment comprises a topic (sources of frustration and choice prevention during the search for medical information on the Internet), a set of four questions which ‘tell a story’ (the Socratic approach), and then requires four simple answers to each question, or a total of 16 answers. The Mind Genomics paradigm is designed to be fast, iterative, provide optimal results, and powerful results terms of a measure of the ability of each answer to ‘drive’ the response, a measure of conscious judgment, as well as measuring response time, a metric which reflects deeper cognitive processing or emotions.

The set of four questions and the 16 answers, four answers to each question, appears in Table 1. The objective is to work with untrained respondents, over the Internet, requiring that the answers be simple, direct, and easy to comprehend. The questions in Table 1 never appear in the test stimuli. Rather, the test stimuli comprise simple vignettes, combinations of the answers, 2–4 answers for any vignette. Each vignette has at most one answer from a question, but for many of the vignettes one or two questions do not contribute an answer. This design structure is deliberate, for statistical reasons, specifically to increase the strength of the analytic tool, OLS (ordinary least-squares) regression.

Table 1. The four questions and the four answers to each question

Figure 1 (left panel) shows the screen shot requiring the researcher to ask four questions. Figure 1 (right panel) shows one question, and the four answers to that question. The answers should be stand-alone phrases that can be understood, in and of themselves. The set-up system for Mind Genomics thus encourages critical thinking, and in the end, a deeper understanding of the topic by combining this thinking with affordable, rapid experimentation among prospective customers.

Figure 1. Screen shots from the set-up of the study. The left panel shows the four questions. The right panel shows the four answers provided by the research to question 3

Experimental design

Each respondent evaluated a unique set of 24 combinations or vignettes. Each vignette comprised 2–4 elements or answers, no more than one answer from any question. The answers were stacked atop each other. The experimental design ensures that for each individual the 16 answers appear several times, and an equal number of times. The incompleteness of the design, with some vignettes absent one or two answers, ensures that the OLS (ordinary least-squares) regression will run without any problem. When the researcher requires each vignette to contain exactly one answer or element from each question, a very common practice, the sad, actually destructive outcome is that the OLS regression can return only with relative values for the coefficients, not absolute values, the cause being the multi-collinearity among the elements due to the requirement that each vignette be ‘complete’ with exactly one answer from each question.

Statistical analysis

The rating scale comprises five points, covering two dimensions, frustration and inability to make a decision. The actual rating scale appears below

Please read the experience below about a person searching for medical information about medical insurance plans. Select which of the following phrases describe the feeling

 1=no problem
2 =not frustrated … Easy to make decision
3=not frustrated … Hard to make decision
4=frustrated … Easy to make decision
5=frustrated … Hard to make decision

We create four variables from this single scale, as follows:

The two ‘negative outcomes’

Frustrate YES (abbreviated as Frust YES) – when the rating is 4 or 5, we code this new variable, Frustrate YES, as 100. Otherwise we code Frust YES as 0.

Decide NO (abbreviated as Dec NO) – when the rating is 3 or 5, we code this new variable, Decide NO, as 100. Otherwise we code DEC NO as 0.

The two ‘positive outcomes’

Frustrate NO (abbreviated as Frust NO) – when the rating is 1,2 or 3, we code this new variable, Frust NO, as 100. It should be obvious that Frust NO is the inverse of Frust YES

Decide YES (abbreviated as Dec YES) – when the rating is 1, 2, or 4, we code this new variable, Dec YES, as 100. Otherwise we code it as 0. Again, Dec YES is the inverse of Decide NO.

The experimental design enables us to run models, at the level of the individual, relating the presence/absence of the 16 elements or answers to either the two negative outcomes, or to the two positive outcomes, respectively. The analysis pools all the data from the relevant subgroup, and runs one entire or ‘grand’ models. The analysis will, however, run individual-levels models for the mind-set segmentation, discussed below in the section on mind-sets.

The equation relating the presence/absence of the 16 elements to the ratings (positive or negative outcome) is expressed by the simple equation: Specific Outcome = k₀ + k₁(A1) + k₂(A2) … k₁₆(D4)

The additive constant, k₀, shows the estimated value of the variable (e.g., Frust YES), in the absence of elements. Of course, all vignettes comprised 2–4 elements by design, so that the additive constant is an estimated value. It can be considered the baseline, the estimated percent of the time one is expected to hear that there a negative or positive experience, even without information about what exactly was presented.

The response time is defined as the time in seconds (to the nearest tenth of second) between the time that the vignette appears on the screen and the time that the respondent assigns a rating. The computer program measures that time. The response time model is written almost in the same way, but without the additive constant. We interpret the model as telling us the estimated number of seconds that the respondent spent ‘processing’ the specific element. The response time is a so-called objective measure, not under the control of the respondent. The respondent may not even be aware of the processing.

Total panel

Table 2 shows the parameters of five models relating the presence/absence of the 16 elements/answers from the four questions to response time (fifth data column) and to four “NET” ratings, those ratings talking about frustration (Frust YES, Frust NO), and those talking about the ability to make a decision (Dec YES, Dec NO). The table is arranged to show the elements which drive the negative outcomes (Frust YES, Dec NO), then the response time, and then the elements which drive the positive outcomes (Frust NO, Dec YES).

Table 2. Parameters of the models relating the presence/absence of the 16 elements to the NET negative and the net positive outcomes respectively, as well as to response time

We begin with the additive constant, which gives us a sense of the percent of responses that will be assigned the NET rating in the absence of elements. Of course, all of the vignettes were created according to an experimental design which prescribed 2–4 elements per vignette, making the additive constant a purely calculated parameter. Nonetheless, the additive constant gives us a sense of a baseline response.

The additive constants for the two negative and the two positive outcomes are the following:

Negative: Frust Yes = 20
Negative: Dec No = 52
Positive: Frust No = 80
Positive: Dec Yes = 48

We conclude that there in general, people don’t feel that they are frustrated with the websites giving information (additive constant = 20 for Frus YES versus additive constant = 80 for Frust NO). In contrast, people feel that they cannot make a decision based upon the website (additive constant = 52 for Dec NO versus additive constant = 48 for DEC Yes).

Five specific answers or elements strongly frustrate the five frustrating elements are:
Information is scattered – frustrates me
Information doesn’t seem trustworthy
Information is relevant but hard to understand
Looking for specific answers
Have to do multiple searches before making a decision

Of these, three lead to aborting the decision
Information is scattered – frustrates me
Information doesn’t seem trustworthy
Information is relevant but hard to understand

Response time tells us additional information, namely the degree to which respondent think about the answer
Looking for process … how to…
Make best decision considering my situation
Looking for specifics
Make best decision for my future
Have to do multiple searches before making a decision
Looking for simple way to contact and get services
Make best decision considering health
Looking for: specific rates
Make best decision
Information is relevant but hard to understand

How problems interact with solutions

A key benefit of the permuted experimental design is the ability to assess the nature of the interaction between pairs of elements [13]. The approach is called scenario analysis. The scenario analysis holds a single element constant from one question, and estimates the coefficients of all elements or answers from the other questions.

The method of scenario analysis was applied to determine how the elements or answers to Question D, information wanted, interacted with the remaining elements. As will be seen in this analysis, the interactions can be dramatic. Depending upon the specific type of information wanted, some elements may be seen to frustrate not at all, or turn around and frustrate a great deal.

The process followed these steps for the first dependent variable, the negative outcome Frustrate YES

1. Sort the 1200 records into five strata, based upon the specific element or answer from Question D. Question D contribute either no answer to a vignette, or one of four answers. We sort the database into the five strata.
2. For each stratum, we estimate the additive constant and the 12 coefficients, A1-C4. We do not use the elements or answers from Question D because they are either absolute, or held constant.
3. The results appear in Table 3 for the dependent variable being Frustrate YES, i.e., the combination of answers where the respondent said she or he would be frustrated, whether or not the respondent would make a decision.
4. We begin with the additive constant, which is very low when there is no information wanted (additive constant = 18). The basic frustration is highest (additive constant = 35) when the respondent is presented with the task of ‘looking for process how to file the claim.’
5. An element can be alternately not frustrating or very frustrating, depending upon what one is searching for. Consider element C2 (Have to do multiple searches before making a decision.) When the respondent is looking for information (specific rates or simple way to get services), there is no frustration. Multiple searches do not lead to much frustration. When the issue specific and concrete (e.g., specific coverage or process to make a claim), the multiple searches becomes frustrating.
6. The scenario analysis provides the researcher with a new tool to understand how pairs of elements interact with each other. The researcher need not incorporate specific interactions ahead of time. Rather, the permutation of the underlying experimental design leads naturally to the emergence of interactions, and an easy way to discover them.

Table 3. Scenario analysis showing how the combination of specific elements with elements from the fourth question (information-wanted) drives Frustrate YES

We now turn to the second dependent variable, the negative outcome of No Decision Made. The results appear in Table 4. The pattern is radically different.

Table 4. Scenario analysis showing how the combination of specific elements with elements from the fourth question (information-wanted) drives Decision NO

1. The greatest basic likelihood of no decision is ‘Looking for specific coverage’ (additive constant = 72.) The next highest likelihood of no decision is ‘Looking for process how to file a claim’ (additive constant = 59). The remaining two elements (Looking for specific rates and Looking for simple way to contact and get answers) show lower additive constants, 40 each. The take-away from this initial finding is that the likelihood of no decision is a function of what people are looking for, with the most problematic being specifics. The website should concentrate on example of specific services, or a way to provide a rate.
2. When there is no task, the additive constant is low (22) but many of the elements drive the decision. The most severe is ‘information doesn’t seem trustworthy’ but there are many other elements which strongly drive ‘No Decision.’
3. There are specific interactions which make intuitive sense, such as Looking for simple way to contact and get answers (D2) coupled with Current disease stage (A3). We might not immediately think of that, but the data reveals the interaction, and suggests that we pay attention to that possible problem combination.

We now turn to the third and final dependent variable, response time. As noted above, response time does not measure a cognitively meaningful response to the vignette such as Frustrate YES, Decide NO, but rather the length of time required for the respondent to process the information in the vignette, and assign a rating. The equation does not have an additive constant, because without any elements there is no predisposition to respond. Furthermore, our focus is on the effect of one of the four searches, D1-D4. We consider only four strata, each stratum fixing one of the four search goals.

Table 5 shows the parameters of the models. The response times are quite long for the individual elements, often longer than 2.3 seconds, the cut-off level beyond which the cell is shaded, and the numbers in bold text. The interactions are different across the four answer for Question D, ‘Information Wanted.’

Table 5. Scenario analysis showing how the combination of specific elements with elements from the fourth question, information-wanted, drives Response Time

Looking for: specific rates
Make best decision considering health
Make best decision
Current disease stage

Looking for simple way to contact and get answers
Make best decision considering my situation
Make best decision
Make best decision for my future
Information is relevant but hard to understand

Looking for specifics
Have to do multiple searches before making a decision
Information is scattered – frustrates me
Information is relevant but hard to understand

Looking for process …. how to….
Current disease stage
Make best decision for my future
Information doesn’t seem trustworthy
Information is relevant but hard to understand
Make best decision considering my situation

Key subgroups

The ability to have each respondent evaluate the precise array of vignettes makes it easy to the researcher to look at different groups of respondents, and at the same time be assured that the pooled data will both maintain the statistical independence need for OLS regression, and cover a large proportion of the design space.

We begin again with the net attribute, Frustrate YES. The additive constant shows dramatic group to group differences in the basic likelihood to say ‘frustrated’ when presented with the vignette ‘without elements’ (see Table 6). When we compare the additive constant with the net attribute, Decide NO, we find radical differences. There is a very wide range of basic levels of frustration, as shown by the additive constant, whereas a much narrower range in the basic inability to make a decision.

Table 6. Additive constants for group-based models relating the presence/absence of the 16 elements/answers to both the Net Attribute ‘Frustrate YES’ and the Net Attribute ‘Decide NO’

Table 7 shows the parameters of the models by each subgroup for Frustrate YES

Table 7. Group models relating the presence/absence of NET Variable Frustrate YES to the 16 elements

Among the most frustrating elements are
Information is scattered – frustrates me
Information doesn’t seem trustworthy

Table 8 shows the parameters of the model by each subgroup for Decide NO

Table 8. Group models relating the presence/absence of NET Variable Decide NO to the 16 elements

Among the elements which hinder decisions are
Information doesn’t seem trustworthy
Information is relevant but hard to understand
Information is scattered – frustrates me

The older respondents (age 50–59 and age 60+) are the most likely to report frustration or inability to make a decision.

The youngest respondents, report that they are frustrated, but they say that they can make a decision. This is an important fact. It appears that frustration may be an emotional reaction whereas decision may be a simple action.

Mind-Sets in the population based upon how easily a person is frustrated in the search

A key feature of Mind Genomics is the extraction of new-to-the-world mind-sets, based upon how the person thinks with regard to the specific topic. During the past sixty years, consumer researchers have recognized the value of dividing people by patterns, either of WHO they age (geo-demographics), what they DO (behavior), or how they THINK about general topics [17].

The notion that people differ from each other is obvious but it is not clear that one can know exactly WHAT TO SAY to a person when one knows WHO they are, what they DO, or what they BELIEVE. One could make the case, of course, that one knows certain things that one should say, but what are the precise words, the precise communication messages for a person, say in the world of health information? How does one know what to say to a person on the web, or the next person who walks in the door?

It is tempting to believe that the general segmentations based upon previous behavior will indicate what to say. The answer is not clear. Behavioral targeting is all the rage today, as of this writing (August, 2019), but it is not clear that a person who asks for rates about a health service will answer every message. The words must be correct. The sensitivity to the mind of the patient must be tuned. And, most of all, one must ‘know’ what words to do, what actions to take, not so much in the grand world at so-called 20,000 feet, where the detail is lost, but rather ‘on the ground’ in granular detail.

Mind Genomics works in the world of the concrete, the world of daily experience, the world of specific words and phrases. The studies in Mind Genomics are not posed as questions to be answers, but as vignettes to which one reacts. This structure eventuates in the above-demonstrated set of coefficients. Segmentation, in turn, becomes the identification of different and meaningful patterns of coefficients, and therefore ‘mind-sets.’ The ‘mind-set’ is a coherent pattern of coefficients, with each respondent in the study assigned to one of the mind-sets, based upon the pattern of that individual’s coefficients.

For this study, the discovery of the mind-sets was based upon clustering of individuals using the coefficients from Frustrate YES, i.e., individual patterns of getting frustrated. Frustration is a basic emotion among people, especially those who search for necessary information. The process follows these steps:

1. Array the coefficients for Frustrate ME so that each row is a respondent, and each respondent has 16 numbers, one for each coefficient. The additive constant is not used for the cluster analysis.
2. Cluster the respondents using K-means clustering [18]. so that individuals close together are clustered together. The measure of distance is (1-Pearson R), with Pearson R taking on the value of +1 when two patterns are identical (distance = 0) and with Pearson R taking on the value of -1 when two patterns are diametrically opposite (distance = 2).
3. The objective of clustering and segmentation is to find a meaningful division of respondents, based upon specific criteria. Clustering is a heuristic in exploratory data analysis, not a hard-and-fast system, although the mathematics are stringent and reproducible.
4. Compare the two-cluster and three-cluster solution. Choose the solution with the lower size only when it is easy to interpret. For these data, the three-cluster solution was easier to interpret, but interpretation is a subjective matter.
5. Three mind-sets emerged from the clustering. The clustering program does not label these, but rather the naming of the mind-is left to the researcher. The typical naming considers the strongest performing elements, and in this study, the additive constant. Table 9 presents the strongest performing elements for each of the three mind-sets.

Table 9. How three emergent mind-sets based on what drives ‘Frustrate YES) respond to the elements in terms of the segmenting criterion, Net Frustrate YES

Mind-Set 1 – Has some latent frustration, as shown by the modest additive constant, 24. Mind-Set 1 gets really frustrated when the search is difficult. The latent frustration emerges from the relatively high additive constant of 24. The two strongest elements drive a high degree of frustration

Information is scattered – frustrates me
Information doesn’t seem trustworthy

Mind-Set 2 has almost no latent frustration (additive constant = 3), but also easily frustrated, and strongly so. The same two elements drive Mind-Set 2 compared to Mind-Set 1, only far more.

Mind-Sets 1 and 2 may be combined to generate a general mind-set which simply wants easy answers, in general.

Mind-Set 3 is altogether different. Mind-Set 3 exhibits quite strong latent frustration (additive constant = 35, wants search to be simple, direct, give the information, gets frustrated when process is not straightforward. Here are the strongest frustrating elements for Mind-Set 3:

Looking for simple way to contact and get answers
Looking for specific service
Looking for rates

When we apply the mind-sets just discovered for frustration to blockers of decision (Net Decide NO), we find that Mind-Sets 1 and 2 are similar in terms of their propensity not to decide (additive constant 58 and 60) whereas Mind-Set 3 is less hindered (additive constant 42.). Mind-Set 3 is more of a perfectionist, with B1 (make best decision) a source of failure to make a decision. Table 10 shows these results.

Table 10. How three emergent mind-sets based on what drives ‘Frustrate ME) respond to the elements hindering a decision (Net Decide NO)

When we move to response time, we find dramatic differences.

Mind-Set 2, which showed the least latent frustration (additive constant = 3), also shows the longest response times. That is, Mind-Set 2 reads everything closely.

Mind-Sets 1 and 3 both show a high level of latent frustration (additive constant = 24 for Mind-Set 1 and additive constant = 35 for Mind-Set 3.). These two mind-sets show shorter response times, consistent with their latent frustration. Mond-Set 3 appears to consider the information in a slightly deeper way than does Mind-Set 1, because its response times are slightly longer.

Finding these mind-sets in the population (The Personal Viewpoint Identifier)

A continuing result from Mind Genomics studies is mind-sets distribute in the population in ways that are unexpected. Everyday experience with the different services suggests that there are different mind-sets or preference patterns for various services. The value of a conversation with a health provider (boot, chat or live) is obtained through the personalization using tailored messaging.

Table 12 suggests that who a person IS does not correspond to the mind set to which the person belongs.

Table 11. How three emergent mind-sets based on what drives ‘Frustrate YES) respond to the elements in terms of Response Time

Table 12. Distribution of mind-sets across geo-demographics and disease stage

During the past several years authors Gere and Moskowitz have developed a set of simple algorithms based upon the coefficients of corresponding elements emerging from the clustering program. Either all or just a limited number of the elements need to be used, but at least eight elements are required. The algorithm uses a Monte-Carlo method to identify those elements which best discriminate between two mind-sets or across three mind-sets, when ‘noise’ is added to the data. The algorithm has been labelled the PVI, the Personal Viewpoint Identifier. The PVI works with individual respondents, identifying their mind-set, and where relevant, returning information to them, either information which is informative, prescriptive, or both. Figure 2 shows an example of the PVI. The PVI returns with the mind-set of the respondent and may provide the respondent or the health maintenance organization with additional information in terms of feedback. The PVI takes approximately 30 seconds to administer.

Figure 2.
The PVI for the study, showing the questions and the two possible answers to each question

Discussion and Conclusions

In a world where individuals are increasing empowered to discover information previously known only to a cadre of specialists brings with it the problem of frustration and indecision. The world of UX and CX, user experience and patient experience, respectively, have developed to address this situation. These efforts did not begin with the world of the Internet, but rather were begun much earlier by psychologists studying the interaction of people and machines, the field of human factors.

The issue of user experience is magnified when we move beyond games and shopping, with momentary risk, to the search for health information on the internet. Looking at the response times shows us the amount of time that respondents take to process the information. The response times are quite long compared to other topics.

The importance of this study stems from the use of Mind Genomics as an easy-to-implement first stage in understanding the user experience. Rather than building out the system and then first testing the system for usability, the Mind Genomics approach can suggest different aspects of what frustrates a person, what prevents a decision and the nature of the person as a user. Mind Genomics thus gives voice to the individual as well, not as a purely linked part of the user experience, but as another independent dimension, based upon the proclivities of the user. As such, Mind Genomics carries forth the vision of Goldsmith [19], who two decades ago recognized the coming tidal wave of innovation. Mind Genomics is just one of the contributors to what promises to be an increasing tidal wave of innovation as the opportunities and problems in managed health care become increasingly obvious with our aging population.

Acknowledgement

Attila Gere thanks the support of Premium Postdoctoral Research Program of the Hungarian Academy of Sciences.

References

1. Fein R (1986) Medical care, medical costs: The search for a health insurance policy. Cambridge, MA: Harvard University Press.
2. Fox S (2011) The social life of health information. DC: Pew Internet & American Life Project 1–33.
3. Arora NK, Hesse BW, Rimer BK, Viswanath K,Clayman ML, et al.(2008) Frustrated and confused: the American public rates its cancer-related information-seeking experiences. Journal of general internal medicine 23: 223–228.
4. Kim K, Kwon N (2010) Profile of e-patients: analysis of their cancer information-seeking from a national survey. Journal of health communication 15: 712–733.
5. Viswanath K (2005) the communications revolution and cancer control. Nature reviews cancer 5: 828.
6. Janeice-Morgan A, Janeice-Mogan M, Trauth M (2013) Socio-economic influences on health information searching in the USA: the case of diabetes. Information Technology & People 26: 324–346.
7. Ayers SL, Kronenfeld JJ, (2007) chronic illness and health-seeking information on the Internet. Health 11: 327–347.
8. Macinnes N,HaglundBJ (2011) Readability of online health information: implications for health literacy. Informatics for health and social care 6: 173–89.
9. Cebul RD, Rebitzer JB, Taylor LJ, Votruba ME(2011) Unhealthy insurance markets: Search frictions and the cost and quality of health insurance. American Economic Review 101: 1842–1871.
10. Box GE, Hunter WG, Hunter JS (1978) Statistics for experimenters. New York, John Wiley.
11. Green PE, Rao VR (1971) conjoint measurement for quantifying judgmental data. Journal of marketing research 8: 355–363.
12. Green PE, Srinivasan V (1990) conjoint analysis in marketing: new developments with implications for research and practice. The journal of marketing 54: 3–19.
13. Moskowitz HR (2012) ‘Mind genomics’: The experimental inductive science of the ordinary, and its application to aspects of food and feeding. Physiology & Behavior107: 606–613.
14. Gofman A, MoskowitzH (2010) Isomorphic permuted experimental designs and their application in conjoint analysis. Journal of Sensory Studies 25: 127–145.
15. Moskowitz HR, Gofman A, Beckley J, Ashman H (2006) Founding a new science: Mind genomics. Journal of sensory studies 21: 266–307.
16. Moskowitz HR, Gofman A (2007) Selling blue elephants: How to make great products that people want before they even know they want them. Pearson Education.
17. Wells WD (2011) Life Style and Psychographics, Chapter 13: Life Style and Psychographics: Definitions, Uses, and Problems. Marketing Classics Press.
18. Dubes R, Jain AK (1980) Clustering methodologies in exploratory data analysis. Advances in Computers 19: 113–238.
19. Goldsmith J (2000) The Internet and managed care: A new wave of innovation. Health Affairs 19: 42–56.

DOI: 10.31038/ASMHS.2019342

Abstract

We present a new approach to understand what draws individuals to romantic ‘dating’ sites. The approach follows a Socratic sequence, requiring the researcher to follow a set of steps. These begin with defining the topic (dating site), then ask six related questions which ‘flesh out’ the topic, and then require six answers for each question, these answers or elements providing the detail. The actual approach is an experiment, in which test participants (respondents) evaluated 48 different combinations of these answers, in short vignettes comprising 2–4 answers. The relation between the individual respondent’s ratings and the presence/absence of the 36 elements allow for a regression model, whose parameters show the contribution of the elements. The process creates a database of knowledge (what do people want), identifies complementary mind-sets, and then creates a personal viewpoint identifier (PVI) which assigns a new person to one of the mind-sets by a simpler set of six questions, and one of two possible answers. We discuss the potential of this simple, rapid, cost-effective, and powerful method to create a large library about the ‘mind of the consumer.’

Introduction

A great deal of the popular press, especially on the Internet, involves the search for relationships, especially between those who would be called ‘potential partners.’ A quick look at ‘relationship dating apps’ in Google comes up with 366 MILLION ‘hits,’ ‘dating sites’ comes up with 704 MILLION hits. Asking for the ‘best‘dating apps’ produces a long list, some which are: Zoosk®, Match®, Elite®, Silver Singles®, and so forth. Moving over to the academic world, with Google Scholar, the repository of academic publications, we see 1.4 MILLION hits for ‘dating sites’ and 33,200 hits for ‘dating apps.’

Of course, we do not have to go to Google® or the scientific literature. We need only listen to the casual conversations around us to know how focused people are on their current partners, past partners, possible partners, and of course the world of non-partners considered from the viewpoint of casual relationships.

To capitalize on this search for partners and this fascination, a variety of companies have been formed to match partners. The applications do not begin as of this writing, or even as of this century, but go back to the 1960’s with computer dating. Indeed, author Moskowitz has had personal experience with these programs as far back as 1966, when the computer was recognized as a device to ‘match profiles,’

The presumed superiority of on-line dating sites comes from claiming that an underlying algorithm is better at matching two people than the traditional methods, such as blind introduction by friends of relatives. Whether, in fact, the algorithm is better than judgment is not the topic of this study. Rather, it would appear that even if the algorithm were not as good as judgment, the dating site comes up with an assortment of prospective partners, making the pursuit of the ‘right partner’ more time-efficient [1] There are a variety of studies of the user of the dating site from different aspects of the person himself or herself [2–6] but these studies do not appear to focus on the dating site as a product to be created, like any other consumer product.

Background

We use the emerging science of Mind Genomics to understand how people respond to the features of an online dating ‘app.’ Mind Genomics is an emerging psychological science dealing with the experimental analysis of the ‘everyday,’ and, specifically, the criteria by which we make decisions [7,8] Mind Genomics is an ‘experimenting science,’ which means it bases its data on the results of experiments in which the respondents are given various combinations of messages, features, ideas, combined in assortments according to a plan, so-called experimental design [9] The objective is to estimate how every one of the different messages of features ‘drives’ a response, whether the response be interest (our first rating question), or selection of a feeling after reading the combination.

The notion of experiment is important in Mind Genomics. A great deal of information about people, about desires in relationships, and so forth, is obtained by direct questions, the survey procedure. The survey answers are tabulated, analyzed through statistics, perhaps correlated with each other and with outside information, until the answers emerge through a pattern. The survey may be attitudinal. Another approach, increasingly popular, observe behavior in a natural setting, such as search information, and establishes patterns from the search information.

Mind Genomics is neither a survey of attitudes and practices, nor an observation of free ranging behavior in a choice situation. Rather, Mind Genomics is an experiment. Mind Genomics presents the ideas in combinations, vignettes, as noted above, deconstructs the response into the contribution of the individual components of the combination, the vignette, and then identifies a pattern from the data. The components, i.e., phrases, are ‘cognitively rich.’ They are statements about behavior which can stand alone and have meaning, the type of information to which we would ordinarily be subjected. It becomes easy to discover new ideas by simply looking at the ‘meaning’ of elements which perform well together in a specific subgroup.

Putting the Mind Genomics approach into action with ‘Online Dating’

The study comes from the interest of the authors in the nature of relationships, and what people want from a technology which creates the introduction. The topic, while not seeming profound, is actually quite deep in terms of the engagement of people who ‘set up the study.’ That is, for many Mind Genomics studies the efforts to create the study are made, but there is little excitement at the beginning. The topics may be profound, but to the ordinary person, the non-expert, the topics are mundane, if not downright boring. In contrast, the group(s) working on this study and similar studies comprised mainly women, typically between the ages of 18 and 50, rather than men. The topic, the questions, the answers (elements), were generated by women.

Step 1: Choose the topic and ask six questions which ‘tell a story.’ The topic as chosen was ‘online dating,’ specifically the features that a person would find interesting, and the emotions that would be experienced when using this online dating. Table 1 presents the six questions, and for each question, six answers.

Table 1. The six questions and the six answers to each question

Combining Elements (Answers To The Questions) Into An Experimental Design

One of the tenets of Mind Genomics is that people cannot easily tell the researcher ‘what is important’ or, in a more quantitative fashion, assign the level of importance of each aspect of a situation. We may believe we can, but our judgment of what we read, what we see, what we hear, feel, smell and taste is so rapid that it is likely that any post-hoc reporting of what was important for the judgment may be a rationalization, an intellectualization of an otherwise rapid, almost automatic process. Indeed, Nobel Laureate Daniel Kahneman suggests that most of our everyday decisions come from an automatic reaction to the situations which confront us, decisions that are controlled by so-called System 1. When we are asked to think in a rational way, even about what has been reacted to automatically, we invoke a slower system, so-called System 2 [10]

Mind Genomics works using experimental design, which combines the elements into vignettes, test combinations, with the property that each vignette comprises a limited number of elements (2–4), with at most one element or answer from a question. The rationale is that the experimental design is useful both for ensuring the downstream statistical analyses by regression, and a bookkeeping device to prevent the juxtaposition of potentially mutually contradictory pieces of information in one vignette.

Each respondent evaluated a set of 48 vignettes, with each respondent evaluating a set of combinations totally different from the combinations evaluated by another respondent. The structure of the underlying experimental design is maintained. The mathematics allows the elements to appear an equal number of times, and ensures that at the level of an individual respondent the 36 elements are statistically independent of each other.

The use of mixtures, and the measurement of subjective response to those mixtures, constitutes a hallmark of Mind Genomics. The research strategy focuses on the response of a respondent to a series of unique combinations, with no discernable pattern, no single ‘story,’ and therefore an array of test stimuli, which in the words of psychologist William James, constitute a ‘blooming, buzzing confusion.’ One cannot ‘game’ this system. The stimuli come quickly, change, demanding simply that the respondent react to the combinations, a response which may start out as deliberate, but quickly becomes ‘automatic’ as the respondent realizes there is no discernable pattern. It is impossible to ‘game’ the experiment. Even the most determined respondent quickly becomes defeated, and returns to virtual automatic responding.

Figure 1 shows an example of the vignette. The vignette shows a four-element combination, with the elements centered to make reading easy. There are no connectives. The ingoing assumption is that respondents will scan the vignette, graze for the necessary information, and assign their rating. The bottom of the vignette shows the actual rating scale, in this example a 9-point Likert scale, anchored at both ends. The design of the vignette makes it easy to go through a large number of different vignettes, without becoming fatigued. Furthermore, the program is set up so that as soon as the vignette is rated on the first scale (joining), the second rating scale comes up (select an emotion, Figure 2). After the respondent has rated the vignette on these two scales (joining, select an emotion), the program automatically presents the next vignette.

Figure 1. Example of a four-element vignette, with the first rating scale

Figure 2. Example of the same four-element vignette, this time with instructions to select the appropriate emotion experienced after reading the vignette.

Table 2. Presents the summary results, sorted by the performance on Question #1 (join)

Running The Study

Upon clicking a link in the embedded email, sent by the e-panel provider, the respondent was presented with the invitation, and the information shown in Figure 3. The information provides very little background about the reasons for the study, other than the statement ‘Today, you will taking a survey regarding Online Dating.’ Although Mind Genomics is an experiment, the term ‘survey’ is less frightening, and was used in order to assuage any fears that might be aroused by the use of the technically more correct word ‘experiment.’

Figure 3. The orientation page, shown to the respondent at the start of the Mind Genomics experiment

There are some specific caveats about the actual material, such as the fact that all of the vignettes differ from each other. In the earlier visions of the program, especially those not on the web but done in person on micro-computers, many respondents stated that they felt they were seeing the same vignette several times. This is a natural reaction, stemming from the fact that the respondent sees the same elements, combined however into different vignettes. They remember some of the phrases, and when they see the phrase again, they may not realize that the other elements in the vignette have changed.

The second noteworthy piece of information is the statement that the study will take 10–15 minutes. Respondents do not like open-ended, boring experiences which seem that they will never end. By giving the respondent a sense of a 10–15-minute experience, we forestall a lot of the potential for the length of the Mind Genomics experiment to become an issue.

A parenthetic note: We should keep in mind that there were no marital requirements for this experiment when the respondents were selected. We could have limited the study to unmarried individuals, but we wanted to get a general representation of individuals, married, single, younger, older males and females, respectively.

Data Analysis By Modeling – Question #1 – Join

Each respondent evaluated a set of 48 vignettes, arranged according to an experimental design. The ratings for each respondent for question 1 (join the dating website) were transformed so that ratings of 1–6 were replaced by 0 to denote little or no interest, and ratings of 7–9 were replaced by 100 to denote moderate or high interest. A very small random number was added to each binary transformation. The binary transformation follows the common practice of researchers in the social sciences and in political polling, who have come to realize that the precision of the scale is not easily translated to meaningfulness when the results of the study are subsequently applied by managers. Most managers, and indeed most researchers, do not know what the scale means, and especially have little or no idea what the intermediate scale points signify. The binary transformation simplifies the communication of the results and the use of the data.

The first column is the element code, the second column is the element text, and the remaining columns are the results from the different groups (total, gender, age).

The base size – number of respondents in the specific group

Additive constant –percent of responses which would be 7–9 in the absence of elements. The additive constant is a purely estimated parameter since all the vignettes by design comprised 2–4 elements. Nonetheless, the additive constant can be viewed as a baseline, showing the predisposition of the respondent to be positive. The additive constants are low, with the younger respondents (age 18–38) most optimistic, and the older respondents (age 53+) least optimistic about the site.

The performance of the individual elements. Mind Genomics provides a rich database from which to extract patterns. Mind Genomics does not need a pattern to ‘jump out,’ with the elements of the pattern simply observations. Rather, each element is ‘cognitively rich,’ with many possible meanings. The patterns which emerge come from both the meaning of the individual elements which score highest, as well as the commonality among those elements.

Total panel – free is the strongest performer:

Put away your credit card…everything on this website is free

Free personality test…matches you with more singles in your area

Male – many different elements, but free and easy are the strongest performers

Free personality test…matches you with more singles in your area

Registration is fast and effortless

Fast efficient match making tools

Put away your credit card…everything on this website is free

Online dating made easy

Receive matches by e-mail, free

Easy login… with Facebook

Females – free, safe (with an emphasis on safe, no risk)

Put away your credit card…everything on this website is free

Over 8,000 singles online now…making more connections everyday

Age 18–38 – Others are involved

Over 8,000 singles online now…making more connections everyday

Age 39–52 – Free, others are involved, the system has a number of assurances of ‘performance’

Put away your credit card…everything on this website is free

Has more dates, more relationships and more visits than any other online dating website

Over 8,000 singles online now…making more connections everyday

Partnership with Match.com to increase our services to you

Age 53+ – Privacy, assurance of ‘performance’

Free personality test…matches you with more singles in your area

For singles concerned with privacy, rest assured contact details are never published

Linking Emlinking Emotions To Elements

An emerging topic of interest over the past decade has been the way emotions come into play when a person reads something or does something, the idea of affect as an accompanying feature of behavior. Often it is difficult to pinpoint the precise emotion, since there are hundreds of words which describe one or another feeling, emotion, disposition. The nomenclature of emotion makes it very difficult to select a set of words, although if a limited set is not chosen, the task becomes virtually impossible.

A parenthetical note: The same issue, description of the qualitative aspect of emotion, can be said to describe the state of our approach to the description of odor and a lesser issue but a problem nonetheless for the description of texture. The issue in emotion, olfaction, and texture, is simply the fact that there are no agreed-upon primaries, and the subjective richness of the emotion, the smell, and the touch can be simply overwhelming.

We recognized this problem, and selected five different emotions, two negative (uncertain, intimidated), one intermediate (disinterested), and two positive (curious, eager.) The emotions were not mean to describe a continuum, but simply designed to capture a variety of feelings, positive, negative, and a variety of actions (information-seeking; approach-avoidance.)

After the respondent rated the vignette on the overall evaluative criterion of ‘join,’ the respondent was instructed to rate how she or he felt when reading the vignette. The five selections became five new dependent variables. When a specific emotion or feeling was selected (e.g., uncertain) the newly created variable corresponding to that emotion was assigned the value of 100. The remaining four newly created variables were assigned the rating of 0.A small random number was added to each newly created rating.

The coefficients for the five newly created variables were estimated, this time again using OLS regression, but without an additive constant.

Table 3, 4 presents the linkages between the 36 elements and the nature of the feeling (both positives, neutral, both negatives, and then the four specific feelings).For the positive, disinterest (neutral) and negative feelings, we show the strongest linkages, shading those of 15 or higher. For the remaining four feelings (uncertain, intimidated, curious, eager) we shade those linkages of 11 or higher. A linkage of 10 is highly statistically significant in the OLS regression model

Table 3. Linkage between the 36 elements and the positive, neutral (disinterest) and negative feelings, as well as the four specific feelings

Table 4. Performance of the elements by the two mind-sets

Table 3 tells us that the majority of strong emotional linkages to these elements are either disinterest or negative. Although there are some elements which may drive an individual to say that she or he will ‘join’ the dating site, there is very little ‘massive’ positive reaction. What positive reactions occur can be traced more to curiosity than to eagerness

Discovering Mind-Sets In The Population

Underlying the Mind Genomics effort is the belief that for any specific topic where decisions are made, people exhibit a range of criteria, or ‘mind-sets.’ The effort may be very broad such as what interests an individual in an organization, or very narrow, such as the topic of joining a data site, or even a narrower aspect of the topic, such as how to pay for the site, and so forth. The key is being able to ask questions, provide relevant answers, and execute the study among the relevant respondents. The topic of ‘relevant’ is an entirely separate issue, and may be limited to individuals who are single (for a data site), individuals who have used a dating site, or even individuals who have used a specific dating site. On the other hand, the relevant respondent may be just about anyone.

In the world of genomics, we deal with a gene, and alternative versions of the same gene, alleles. The comparison to Mind Genomics is metaphoric. Whether the respondents are from a specific group or from a general population, they nonetheless often divide into groups with different points of view, so-called mind-sets, mental genomes. The experiment we have just run in the above-discussed results provide us with the tools to isolate mental alleles, mental variations of the same ‘gene,’ the gene being the ways one thinks about joining to a dating site.

Discovering mind-sets in the population of respondents combines objective statistics and subjective interpretation. Each respondent generated a model relating the presence/absence of the elements to the binary value of 0/100 (0 when the original rating was 1–6, 100 when the original rating was 7–9). The binary-transformed ratings were then used as the dependent variable and related to the presence/absence of elements. The coefficients (but not the additive constant) were used as inputs to a k-means clustering program (Jubes& Jain, 1980). The program divided the respondents into two groups (clusters, mind-sets), and then again into three groups. The criteria for dividing the respondents was mathematical in nature. The selection of the clusters, two or three, was based upon a simple pair of criteria, parsimony (fewer clusters are better than more clusters), and interpretability (the clusters must ‘tell a coherent story.’)

For these results, it was easy to interpret the two-cluster solution, suggesting two radically different mind-sets. Table 3 shows the coefficients for the two mind-sets. Both start with low additive constants, 21 and 22, respectively, suggesting that in the absence of the elements about one-fifth of the time the rating will be 7–9. This means that it is the task of the elements to drive ‘join.’ In neither mind-set are people ready to join.

The richness of the elements in terms of what they convey, how they represent the way a person feels, and the simplicity of the pattern reflects the nature of Mind Genomics. Rather than extracting a pattern from points, and thus forcing oneself to see order in possible chaos, Mind Genomics extracts a pattern from phrases which themselves are meaningful. One need not even ‘name’ the mind-sets to understand the results. Just knowing the high-scoring elements may often suffice, although it is in the nature of researchers to label groups, if only as a aide-memoire.

Mind-Set 1 responds to ‘make it easy and free.’

Over 8,000 singles online now…making more connections everyday

Put away your credit card…everything on this website is free

Mind-Set 2 is a ‘serious dater’

Find G-d’s match for you

Establishing The Consistency of The Ratings From Individual Respondents

The question is often asked whether the respondents in these Mind Genomics studies are serious in their participation, or whether they just answer at random. As of this writing (Summer, 2019) the issue of survey fraud, especially with online studies, has come to the fore as a major topic, especially because survey fraud can be committed with ‘bots’ programmed to answer surveys as if the bots were legitimate respondents.

One way to determine whether or not the respondents were actually focused on doing the project uses a measure of ‘goodness of fit’ of the equation to the data, the multiple R-square. The individual-level regression model, so-called curve-fitting model, OLS (ordinary least-squares) regression, may or may not ‘fit the data. ’When the regression model fits the data, the R-squared is high. The R-square ranges from a high of 1.00 when the regression model fits the data perfectly, to a low of 0.0 when the regression model essentially fails entirely, and is simple a ‘best fit’ but is really irrelevant,

Figure 4 shows the distribution of R-square values for the respondents, each respondent generating a single value from the model generated by her or his ratings, versus the presence/absence of the elements. R-square values of 0.25 or higher (R>0.5) suggest that the respondent is taking the study seriously. Figure 4 shows that most respondents generate reasonably high R-square statistics for their individual models.

Figure 4. Distribution of R-square values by mind-set, for the individual models relating of question 1 (‘Join’) vs the presence/absence of the 36 elements

Finding Segments In The Population

It is easy to classify people by who they ARE or what they DO. These classifications can be quickly obtained from many commercial sources. Once can also define a person by one of the many different classification schemes such as PRIZM® by Claritas, Inc. in San Diego, CA. What is almost impossible, however, is to identify the likely membership in a mind-set that is relevant to a specific, local issue such as dating. How can one assign a new person to Mind-Set 1 or Mind-Set 2 for a dating site?

Recent developments by author Gere have evolved into a method which assigns new individuals to the two mind-sets, using a simple algorithm. The approach, PVI (personal viewpoint identifier), is a simplified algorithm, taken in part from the more powerful method of discriminant function analysis [11] DFA requires the raw data, whereas the PVI uses the average data to identify a set of questions, which best differentiate among the different mind-sets. The PVI is a Monte-Carlo process, which adds ‘noise’ to the coefficients to simulate ordinary variation, and identifies the best set of questions and the pattern of answers. The output of the process is the set of questions (usually six), each with two answers, and thus 64 possible patterns of answers. The pattern of answers defines mind-set membership.

Figure 5 shows the PVI. Once the respondent participates in the PVI, the program automatically assigns the respondent to a mind-set, and stores the data. The PVI can be programmed to send feedback to the respondent based upon the pattern of responses, and the mind two which the participant in the PVI is assigned.

Figure 5. The PVI (personal viewpoint identifier) for the dating site

Discussion And Conclusions

The sheer volume of interest in dating sites drive researchers to focus on the site as a way to understand the motives of people, their feelings, and their behavior. In turn, business people recognize the great deal of money to be made from these dating sites because those who are on the sites are there for emotional reasons, whether sincere or feigned, whether looking for connection or for something else.

Mind Genomics brings a different focus to the issue of dating sites. The sites are interesting as a product that can be designed, as well as a micro-topic of everyday life which is of great interest to people in the world of always-connected, and the ubiquitous Internet. The consequence of a new world of interconnections through technology, and increasing social isolation of people because of the advances in technology which have destroyed the tight-knit local communities, make a perfect matrix of research opportunities for Mind Genomics to look at various aspects of dating sites. This first study looks at the site as a business opportunity, using research questions and probes typically encountered in the development of products in the world of commerce. Further studies might well focus on issues of emotionality when the site is used. These further studies would represent the application of Mind Genomics to more traditional aspects of dating sites, the aspects looked at by sociologists and psychologists.

Acknowledgement

Attila Gere wishes to acknowledge and thank the Premium Postdoctoral Research Program of the Hungarian Academy of Sciences

References

1. Finkel EJ, Eastwick PW, Sprecher S (2012) Online Dating: A Critical Analysis from the Perspective of Psychological Science. Psychological science in the public interest
2. Blackhart GC, Fitzpatrick J, Williamson (2014) Dispositional factors predicting use of online dating sites and behaviors related to online dating. In: Computers in Human Behavior Elsevier.
3. Kim M, Kwon KN, Lee (2009) Psychological characteristics of Internet dating service users: The effect of self-esteem, involvement, and sociability on the use of Internet dating services. CyberPsychology&Behavior 12: 445–449.
4. Sautter JM, Tippett  RM, Morgan SP (2010) The social demography of Internet dating in the United States. Social Science Quarterly 91: 554–575.
5. Stevens SB, Morris TL (2007) College dating and social anxiety: Using the Internet as a means of connecting to others. CyberPsychology&Behavior 10: 680–688.
6. Valkenburg PM, Peter J (2007) Who visits online dating sites? Exploring some characteristics of online daters. CyberPsychology&Behavior 10: 849–852.
7. Moskowitz HR, Gofman A, Beckley J, Ashman H (2006) Founding a new science: Mind genomics. Journal of sensory studies 21: 266–307.
8. Moskowitz HR, Gofman A (2007) Selling blue elephants: How to make great products that people want before they even know they want them. Pearson Education.
9. Box GE, Hunter WG, Hunter JS (1978) Statistics for experimenters  New York, John Wiley
10. Kahneman  D,  Egan P (2011) Thinking, fast and slow. New York: Farrar Straus and Giroux.
11. Crask MR, PerreaultJr WD (1977) Validation of discriminant analysis in marketing research. Journal of Marketing Research 14: 60–68.