DOI: 10.31038/PSYJ.2021342

Abstract

The paper presents a novel, cost-effective, data-driven, rapid and scalable way to deal with crises in any industry where the crisis is the result of easily swayed consumer opinion. We introduce here a system which allows anyone to understand the complexities of a problem by dimensionalizing the problem into four relevant questions, and four answers to each question. This dimensionalization is accomplished through an interactive ‘consulting chat’ which guides anyone through to the deeper understanding by the foregoing deconstruction. The deconstructed elements (answers to the questions), are combined into vignettes, combinations of the answers, presented to a small, cost-effective group of consumer respondents through the web, who rate the combination in terms an aspect deemed relevant (here, the feeling towards the food industry, either negative or positively, respectively). The results are automatically and immediately analyzed, to reveal the contribution of each of the 16 answers, sending the researcher a user-friendly, presentation-ready data in an Excel format. The presentation provides a full analysis of the data, showing the contribution of each answer both to the rating (cognitive), and to the response time to the vignette (non-conscious physiological measure). The presentation further shows the contributions of each answer to the ratings, first by total panel, then by gender, then age, and finally uncovers both two and three new ‘mind-set’ segments, possibly unknown previously, with these segments defined on the basis of patterns of responses to the 16 answers. The approach is illustrated by simple, easy-to-implement study on the controversial topic of the ‘food industry’s responsibilities in the obesity crisis.’

Introduction – lies and today

Proliferating media, and today’s increasingly simple ways to send messages, are mixed blessings. Certainly, information gets passed around. And, in the case of truly helpful information, an increasing number of people have an opportunity to get to that helpful information. The result is an increasingly interconnected world, with the blessing of communications that can be shared to help people.

Today’s communication environment is not, however, an unmixed blessing, as the topic of ‘fake news’ continues to reveal, day after day, local crisis after local crisis. An aphorism often attributed to the pundit and writer Mark Twain, but probably more likely originating with that arch cynic, Jonathan Swift, is ‘a lie can travel halfway around the world while the truth is putting on its shoes.’ In Swift’s original statement, more than three centuries ago, written in the Examiner of 1710.

Besides, as the vilest Writer has his Readers, so the greatest Liar has his Believers; and it often happens, that if a Lie be believ’d only for an Hour, it has done its Work, and there is no farther occasion for it. Falsehood flies, and the Truth comes limping after it; so that when Men come to be undeceiv’d, it is too late; the Jest is over, and the Tale has had its Effect…

(Wikipedia, https://quoteinvestigator.com/2014/07/13/truth/)

How does one combat lies, disinformation, or today’s popular phrase, Fake News? One cannot stop the issue at the source. Yet, it might be possible to fight using information derived from the mind of the citizen, uncovering just what messages are relevant, and just what messages are believable. That is, knowing the mind of the audience ahead of time or during the time of crisis may provide the necessary arms by which to defeat false information, not by power, but by enhanced, knowledge-driven persuasion.

Experiments instead of opinions

The thesis of this paper is that professionals in an industry can prepare for the often-unexpected onslaughts of bad news, fake news, and outright lies by doing their homework ahead of time. The homework begins by identifying the topics which could become the center of controversy. When the time is appropriate, and the situation warrant, all that is needed is an iterative set of small, affordable, focused ‘experiments’, studies on the response to messaging. The messaging deals with the topic. The experiments produce solid knowledge, insights, but beyond the general insight, specific phrases to use, and specific phrases to avoid.

The goal of the experiments is to identify simply ‘what really works’, what produces the ‘right persuasion,’ not just nice to know facts. The approach presented here comes not from today, but from an aphorism of the great doctor, Louis Pasteur, who opined ‘chance favors the prepared mind.’ A modern corollary of this aphorism might be ‘Knowledge casts a light, cauterizing the ghosts of opinions which infect the shadows.’ (Source author HRM)

The notion of being prepared for negative situations is not a new one. It lies at the heart of every expertise. “Practice makes perfect” and other aphorisms, or perhaps platitudes, too numerous to count, are taken as truth, and they are. What is new in this paper is the use of experimental designs of ideas, to ‘dimensionalize’ issues and situations, identify problems, and identify different statements and counterstatements that could be made, all within an empirical, defensible, cost-effective framework. The notion is iterative experiments as tactics, not experiments as sources of grand knowledge. In other words, experiments to answer the issues of the ‘here and now’, in the same frame, ‘here and now.’

The origin of the ideas comes from two sources. In the mid 1990’s, author Jeffrey Ewald began to work with IdeaMap®, the precursor of today’s Mind Genomics. An ‘Early Adopter,’ Ewald explored the use of systematic study of arguments and refutations, first in business with the objective of the client selling something, and then with the objective of the client having a ready-made set of tested communications to use in a crisis.

The second source was legal research done by author Moskowitz and colleagues, at the behest of author (Professor) James Wren of the Baylor State University School of Law. With several colleagues, including R Rex Parris, a noted trial lawyer, Wren encouraged Moskowitz to work with law cases, reducing them to message components, and then test combinations of these messages as arguments in a law case, in order to identify which argument drove the response of the jury in the desired direction. The efforts eventuated in a recently published book on Mind Genomics and the law [1]. Further, Parris has used the approach to win in law cases, including one substantial award of several hundred million dollars, which he attributed to this use of experimental design of ideas [2].

Demonstrating the approach in a vertical – the world of food

We all eat. To the average citizen, food is good, food companies are bad, news about food is available everywhere, often simply interesting content such as favorite flavors of the season or cooking shows with notable personalities. And then there is the not-so-good side. The food companies are under siege for presumably making us obese, for hooking us on salt, sugar, and fat, for destroying the quality of tomato, especially the tomato flavor for the tomato to last longer and shipper better.

The list is endless. The food industry has many issues to face, more coming almost each day. We have to eat. Food is, for the most part, perishable. Little animals and plants like to attack food, ruining it during their effort to survive by poaching on what should be the domain of human beings. And, since food is necessary, and is really for all living creatures, the adaptations we have made to get and consume food manifest themselves in a society where food plays many important roles, some of which are grist for the mill of social and health issues. And so, the never-ending, swirling controversies around food, new ones cropping up every day.

Introducing the steps

Our focus here is on affordable yet ‘industrial-grade’ production of knowledge about the mind of the citizen, specifically what messages convince the citizen about a topic, and what messages do not convince the citizen. It is vital that we create a system to generate the necessary information easily, rapidly, at a low cost, and in a format that can be used by industry sectors and their associated PR firms, not to mention professionals and students alike. Most important, however, is that the information be valid from a rigorous, scientific perspective, and that the approach must be, like Caesar’s wife, above reproach.

To accomplish our goals, we turn to the experimental design of ideas or messages, known by the rubric of Mind Genomics. Mind Genomics has been vetted by peer review in a variety of different areas [3]. The science of Mind Genomics furthermore traces back to well accepted methods in experimental psychology, specifically the pioneering work of the late R Duncan Luce in mathematical psychology [4], and in the ongoing work of Norman Anderson [5]. A Google Scholar® search of the terms ‘conjoint measurement’ and ‘conjoint analysis,’ will bring up many papers of a refereed nature, reaffirming the acceptance by both the academic and business communities.

At its inception, conjoint measurement was a labor-intensive approach. It would soon be simplified and expanded by Wharton Professor, the late Paul Green, in the 1970’s through the late 1990’s [6, 7]. The vision of creating a fast system, self-authoring, inexpensive, and powerful, was introduced by Moskowitz and colleagues at the start of the 21^st Century, and the rapid adoption of the Internet [8].

The actual steps have been developed to fit into the form of a smartphone APP, a symbol of today’s focus on fast, easy, connection, and democratic. Thus, as we go through the first example (with results) in detail, the reader will be able to see how the thinking behind the problem immediately transfers into steps that one follows to bring the project to life. We illustrate the approach using an example from the issue of ‘obesity,’ motivated by the oft-heard canard that somehow the food industry is causing obesity by its practices. The fact that obesity is increasing, is clear from statistics, and well-accepted. That is a ‘fact on the ground’. What is not true, however, is the canard. How then do we fight it with a strategy and with knowledge and data from consumers?

Introducing the process and illustrating how it is tailored for everyday problems

Step 1 – Rethink the notion of the scientific ‘project,’ and look at the effort in service of an issue

We all eat. Many are fanatic in their pursuit of healthier lifestyles. As the world uses machinery, cutting down on the expenditure of calories, the natural course of people is to consume more than they need, leading to food-based problems, such as obesity. The food industry, people, orientations about lifestyle, and an increasingly contentious, better-informed, information-swamped, less critical thinking public all combine for a perfect storm. The storm, to mix metaphors, generates a lot of heat, controversy, sometimes light. And all too often a mentality of jousting, fighting, and perhaps most distressing, ‘grandstanding’ in the name of something, whether that something has value or not.

We deal with one topic in depth, obesity, showing what can be done in a matter of a few hours, at low cost. We show how the scientific process need not be long and ponderous, need not be a process which requires months and years of expertise, but rather can be a disciplined way to create the necessary knowledge. The rationale is that we want to show how very straightforward it is to run a relatively simple experiment (or several iterative experiments) in just a few hours per experiment, in order to prepare for onslaughts in the media, whether that is broadcast media, text media, or social media.

There is a subtext to the choice. We are writing this paper as a demonstration that almost anyone can use knowledge to properly address issues raised by adversaries, whether interviewers looking for sensationalism, or simply groups of people putting in potentially incorrect, even inflammatory material. Furthermore, we take our ‘own medicine,’ here, setting up a study easily (in 20 minutes), and running the study. We also show how the study can be set up as a ‘chat,’ further making the science available to those who don’t even need to realize that they are ‘doing science.’

We also show the type of information one can obtain from so-called convenience sample, small groups of individuals, easy-to-find on the internet, or to recruit using a so-called consumer panel supplier. We show that the knowledge obtained comes not from the proper sampling of people (confusing people with the ideas that they have), but rather from identifying clusters of ideas (mind-sets), using the people almost as simply the carriers of mind-sts. In other words, using the language and metaphor of genomics, we are interested in the mental genomes, not in the body which happens for the moment to be carrying the mental genomes.

Finally, we are not presenting the well thought out, magnificently produced set of brochures in the way well-established, richer corporations do. Rather, we are appealing to those in a cash-starved environment, with no resources, no magic ‘white knight’ riding in at the last minute to save the day.

Step 2 – Describe the tool which allows us to create knowledge quickly

The tool to be used is BimiLeap (www.BimiLeap.com). BimilLeap is a reduced version of a larger technology known as Mind Genomics. Mind Genomics, in turn, is a statistics-based research tool, used to conduct experiments in which respondents are presented with systematically varied vignettes (descriptions of ideas of situations), and rate the vignette on a defined scale [9].

Although this instantiation of Mind Genomics appears to be nothing more than a plain-vanilla survey, the reality is that the Mind Genomics interview is really an experiment. The respondent is presented with a set of systematically varied vignettes, the variation being the selection of what individual messages are combined in the vignette to create the totality. By systematically varying the composition of the vignettes, and having respondents rate each vignette as a single test concepts, the subsequent statistical analysis (regression analysis) reveals the contribution of every element or independent variable to the response.

The experiment design embodied in a Mind Genomics study dovetails well with the topics being explored above. The topic is made explicit and concrete by asking four relevant questions, which tell a story. The relevant questions, in turn, are answered by four separate statements for each question or a total of 16 answers or ‘elements.’ The selection of the topic, the four questions, and the four answers to each question remains in the purview of the researcher. Although one is forever plagued by the nay-sayer’s aphorism ‘garbage in, garbage out,’ the reality of the exercise is that one is forced to think. Not everything is garbage, and indeed, with practice, one learns to think critically, with the data upon repeated practice, suggesting ‘less garbage, more substance.’

Step 3 – Example: Addressing the contention that ‘The Food Industry Causes Obesity’

An instructive way to appreciate the approach instantiated by the APP is through a case history of an actual problem. The study that we discuss here was presented to a large audience at the Chicago 2018 Annual Meeting of the Institute of Food Technologists (IFT). The session topic was how to communicate as professionals.

The rationale for the topic came from the ongoing attacks faced by the food industry. The food industry is often accused as active participants in the growth of obesity world-wide. The rationale for such accusations ranges from anger at the R&D efforts which are believed to use any ingredients which make economic sense, to marketing which is accused in the sensationalist press of communicating misleading information to consumers. And then there are those who argue that there is, in some unwritten way, an implicit contract between the company and its consumers for the company to take an active role in the consumer’s health.

How then does the BimiLeap, work in such a situation, especially in the hands of novices, who do not have the resources of a corporation behind them, a corporation with trained lawyers, a well-paid public relations firm, an advertising (or several) advertising agencies, and a legion of consultants? What happens when the research effort to build the requisite knowledge must be accomplished, from start to finish, in two hours or less. How can this be done, with a reasonable number of respondents (30 or more), each participating in an experiment lasting 4-5 minutes?

Observation from dozens of different studies suggest that the selection of the topic is easy. What becomes difficult is the following set of steps, after the topic is introduced.

It is important to keep in mind that the study that we report was set up in less than ½ hour, was easy to implement on the web, could have taken as little as one hour to complete had we used an easy source of respondents rather than asking people to volunteer for free. The study, when completed, generated a full report Excel, within one minute of the end of the study. The Excel file was emailed to the researcher (author Zemel), within two minutes after the end of the study.

Step 1 – Set up the ‘contents’ of the study (Table 1, Figures 1-5).

The set up comprises selecting a name for the study (viz., the topic), creating four questions, and for each question providing four answers. Table 1 shows the questions as one might fully ask them. Figure 1 shows the way they are recorded in BimiLeap (www.BimiLeap.com).

Table 1: The four questions and their answers. The table shows the four questions in actual question format.

Figure 1: Define the name of the study, create four questions, and for each question provide four answers. The figure comes from the report, automatically generated at the end of the study, with all of the relevant set up slides captured as part of the research documentation.

Figure 2: Create a third classification question (the first two are age, gender), create an open-ended question for the respondents to complete, and create an orientation page.

Figure 3: The rating scale (left panel). Final thoughts of the researcher (middle panel), and number of respondents desired for the study (right panel).

Figure 4: The respondent experience showing the classification question (left panel), and then one of the 24 vignettes shown as it would appear on a smartphone (right panel).

Figure 5 and 6: The Tino Space Chat. The figure shows the interchange between the researcher setting up the study and the chatbot. The objective is to make the set-up an interactive experience which simulates coaching.

It is as this stage that one is forced to think in a creative way. The selection of the topic, obesity and food, is straightforward. It is not hard to think of a topic, or even to refine our topic. The difficulty begins when it becomes time to ‘structure’ our inquiry, by asking four questions, and providing four answers to each question.

The first ‘stumbling block’ is the decreasing ability to think in a ‘systematic fashion’ We are not programmed in our education to think by asking a set of structured questions, although in reality that’s the easiest way to learn. A child asks questions. Despite the difficulties encountered by the researcher, it is vital that the researcher ‘stick with the script,’ and provide four questions. We simply require that the sequence of questions tell a story.

Figure 1 (middle panel) might have an actual question posted, such as ‘what is the advertising policy of the company?’ rather than just the word ‘advertising.’ Framing the question as a question versus framing the question as a simple statement makes no difference to the respondent, who will only see combinations of answers, and never the see the questions. The role of the questions is simply to facilitate the creation of answers.

From the point of view of science and education, it is becoming increasingly clear that we think easily in general terms, concepts, but have a hard time putting concrete ideas against those general terms. This step, questions, requires thought about STRUCTURE. We found formulating the four cogent question, to be modestly difficult for the researcher. Once the researcher formulates the four questions, a lot of the hard work has been done.

The four questions in Figure 1 are shown as one word each (ingredient advertising, prevention, responsibility, respectively.) The questions are short because they were selected by author Zemel, who had extensive previous experience. Instead of asking ‘what are the ingredients being discussed?’ Zemel simply put down the word ‘ingredients,’ knowing full well that the question is only used to generate the answers. The respondent never sees the underlying questions, but rather only sees the answers. The questions are selected to drive the production of the answers.

For each question, create four phrases which answer the question in different ways. The four answers pose a different challenge. Now, the task is not to think about structure, but rather to think about CONTENT. During the development of the BimiLep APP, it was first unclear how to generate this content. Putting the exercise into the form of question and answer made sense. The next issue was to determine the nature of the answers, how to make them readable, because it would be the answers, more correctly it would be the combinations of answers that would constitute the test stimuli.

The early efforts with BimiLeap ended up with researchers producing long-winded answers, complicated phrases involving different twists and turns of thinking. Others were simple one-word or two-word statements, i.e., very terse responses with little content. Neither type of ‘answer’ worked well. It became obvious that we need to specify how to write an answer. We came up with the suggestion that the answer be single-minded, and no longer than (approximately) 12 words. Formulating one’s thought in this structured manner called into play what might be called one’s ‘mental editor.’ No longer was the researcher grasping for structure in thinking, but rather now forced to provide answers selected with a focus on clarity of expression.

Advertising executives will recognize this step as one of the ways to create good ‘copy,’ i.e., good advertisements. Often, this creative step is described by the phrase ’problem-solution,’ namely pose a problem and then present a solution to that problem.

This step moves beyond the creation of a single, well-executed advertisement, or more correctly, a single, tested, vetted ‘concept,’ from which the advertising is to be created. Instead, the objective of this step is to create a bank of messages to be presented to the respondent in small, easy-to-read combinations. This first step, before creating the combinations, is to create the messages themselves, in a way which allows virtually ANYONE to do the basic creating. The question-answer format has been found by the authors to make the task easier. The question-answer format does not necessarily create brilliant ideas that will convince, but rather the format democratizes the process of coming up with material

Typically, consumer research studies using these approaches allow the researcher to ask many questions about who the respondent is, what the respondent believes, how the respondent has behaved with respect to the topic. The opportunity to acquire so much information about the respondent from the interview often ‘backfires,’ leading to data paralysis. The prevailing, typical attitude is ‘let’s not rush in and get the wrong information.’ Faced with the plethora of choice which potentially might produce analysis paralysis, the speed of the process is jettisoned in favor of ‘getting everything we can.’

BimiLeap was designed to be fast, modestly flexible, and inexpensive. The objective is to identify the ‘answers’, also known as element, which perform best, rather than acquire as much information about the respondent as possible. As of this writing (summer, 2021), BimiLeap acquires only three pieces of information about the respondent (age, gender, and a third question left to the researcher to select). The focus is clearly on the answers, the elements, not on the acquisition of extraneous information which ‘might somehow be useful.’

Figure 2 shows this third classification question. It is a sacrifice, of course, to leave information un-gathered, but it’s more important to have the study run quickly, and to recognize that deep knowledge will emerge when the researcher can do several of these small-scale studies, rapidly and affordably, building knowledge in a sequential, empirical, structured way. And, of course, create a library of responses to ideas, a library that can be searched easily to create even new constructs based upon the pattern of data from a set of somewhat related studies, dealing with one topic, or with a set of related topics.

Figure 1 shows how the app forces the researcher to ‘think through’ the issue by defining the topic, creating four questions, and then as an example, for the question regarding responsibility, provide four answers. BimiLeap forces the researcher into a structured way of thinking, beginning therefore with an undifferentiated issue, then a differentiation forced by the questions, and finally a further differentiation forced by the four answers to each question.

Once the questions are asked and the elements generated, the next steps create a rating scale (Figure 3, left panel), write instructions to have the researcher give some private, archival information about why the study and experiment were created (Figure 3, middle panel), and finally choose whether the study will be run free with data available to the world, or whether the study will be privatized, so that only the researcher knows the study and the data (see Figure 3, right panel).

Rating Scale: It is the rating question which allows the respondent to share his or her feelings. The objective of our project is to obtain a quick, almost intuitive response to the situation or the prospective client, which in this case is the world of food companies. A simple, rating question is:

 Based upon what you just read (the vignette). Please rate your feeling on the scale below

1=Feel very negative about the food industry … 9=Feel very positive about the food industry

Privatization: The privatization option was created to make the approach attractive to researchers who wanted to keep their efforts private, yet still wanted to use BimiLeap to create the data. There are always several constituencies when it comes to research. Everyone would like the research to be free, and instantly available, such as source material from Google about all sorts of information, as well as so-called Big Data. No one necessarily realizes that data ‘cost.’ The plaint is that ‘after all, the data are out there and freely available. How can you charge me for what I can get for free?’ The foregoing plaint is very real, altogether too common, and destined to produce stillborn, meager results if followed. Privatization addresses the need to satisfy the constituency which wants to do the research for their own interests. The default position of BimiLeap is that the data are public, when not ‘privatized.

The policy of BimiLeap is to be free for anyone to use, with the stipulation that the study is ported to the web, and parts available for anyone to see. One can purchase the study results in their entirety for a nominal sum. If the researcher wants to keep the results entirely private, the researcher need only select the ‘privatize option.’

Experimental design, test vignettes, and the respondent experience

We are accustomed to survey research, which asks a question, gets one of several possible answers, and analyzes the frequency with which each answer is provided. The results of this exercise give us a sense of what the respondent thinks about a topic, when asked a single-focus question.

Our experience here was also instructive. Many of us who have come from traditional backgrounds in testing products, but also in testing concepts and advertisements, are accustomed to the conventional practice of presenting one test stimulus, after which the respondent rates that test stimulus on a dozen or more scales, or rating attributes. What happens when one has many stimuli, but only one rating scale? The reason for the many stimuli is obvious; it is in the pattern of responses to the stimuli where the real knowledge lies. And the reason for only one rating scale is that with 24 test vignettes and a limited time, with unwilling or uninterested respondents, it is better to have a short interview than a long one.

Mind Genomics and BimiLeap follow a different path because they emerge from the heritage of experimentation, not the heritage of surveys. The respondent is presented with a set of elements, in our case a set of answers to the questions. The questions do not appear. The task of the respondent is to inspect this seeming random combination of messages (viz.,, answers, elements), and assign a single rating to the combination. The task is a bit jarring at first, because the respondent tries to be ‘accurate,’ carefully reading each of the combinations, the so-called vignettes. The combinations comprise different types of information, often information which may be somewhat contradictory, and certainly not put together in the most felicitous prose. Nonetheless, the respondent’s task is to read the test combination, preferably quickly, and assign a ‘gut-level’ reaction, an unintellectualized response.

The effort to force respondents to evaluate these disparate combinations of answers, our test elements, often produces a nervous respondent. Some complain that they don’t want to participate in a study where the ‘test stimuli do not make sense.’ They drop out. The remainder, most participants from a paid research panel, do not drop out, and end up responding in the desired ‘gut-level’ or intuitive fashion.

The aforementioned experimental design dictates the 24 combinations of elements, with the property that the 24 combinations comprise 2-tuples of elements 3-tuples, and 4-tuples, so that each element (i.e. answer) appears equally often, and that no vignette ever comprises more than one answer from the same question The 16 elements, embedded in the 24 vignettes, are combined in a way which makes the 16 elements statistically independent of each other, permitting regression analysis to be performed on the ratings, to relate the rating to the presence/absence of each element.

Finally, each respondent evaluates a different set of vignettes. The underlying experimental design is the same, but the combinations, the structure of the vignette is such that each respondent evaluates a unique set of combinations. This is called a ‘permuted design’ [10, 11]. The permuted design allows the research to cover a wide array of possible combinations, so one need not know anything about the topic, and yet the experiment will quickly reveal the important versus the unimportant elements.

Before the respondent begins to evaluate the vignettes, it is important to tell the respondent a little bit about the study, but not much. The introduction is the orientation page which tells the respondent about the study. The orientation page should be short, to the point, without giving information which could lead the respondent to answer one way or another. The orientation simply tells the respondent the least amount of information. Figure 4 (right panel) shows the orientation, at the top of the screen, and a specific vignette, or combination of elements below, with the rating scale at the bottom.

In terms of the actual user experience, today’s panel participants have moved beyond paper and pencil, and have even moved beyond the traditional computer screen, and onto a smartphone. Smartphones are ubiquitous. In light of the use of smartphones as both communication devices by phone, and as browsers with which to text information and to traverse the net, we created BimiLeap so that it would perform well when the respondent was browsing on a smartphone. Indeed, in today’s world, it’s not even clear whether consumers have computers. It’s best to allow the interview to occur on a smartphone, on the screen. Figure 4 shows the respondent screens, set up for a smart phone.

Improving the set-up experience by means of Chat

initial experiences with BimiLeap suggested that the actual process would be fairly simple. That suggestion turned out to be more optimistic than was the case. The early uses of the APP were made by individuals with a great deal of experience in setting up studies for Mind Genomics. When others began to use the APP it soon became obvious that their experience was far less smooth than was thought at the start of the process. It became obvious that a novice researcher, doing the Mind Genomics process for the first time was facing a host of problems, most of which were more complex than frustrating than had been the case before.

The problems encountered by those setting up both the obesity study and others ranged from hard-to-understand instructions about what it means to ‘ask a question,’ to ‘provide an answer,’ and even to create rating scales and open-ended questions. Quite simply, what appeared simple to experienced researchers required ‘coaching,’ and perhaps even more.

Author Savicevic suggested that the process move in a direction made possible by a chat bot. We are all becoming increasingly with chats, which engage in a simulated conversation. The chat was designed by the staff at Savicevic company, Tino Space, to act as a more intuitive, chat-based acquisition. Figures 5 and 6 show the set-up process, this time by a chat. The chat set-up, now in refinement, appears to be a substantial improvement in the user experience, at least for the researcher who has to do the set-up work.

The researcher experience – automatically-analyzed results in presentation-ready format (Figures 7-8)

Figure 7: Full report in PowerPoint format.

Figure 8: Information from the Excel file.

Today’s world operates on bites, small bits of information, presented in an entertaining fashion, or at least in a fashion which allows the information to be assimilated quickly. Rather than reading papers, even in areas which are very relevant, many people prefer to have the information presented in a manner that is exemplified by PowerPoint®. That is, rather than digesting the information for themselves from detailed text requiring thinking, many people prefer to have information ‘spoon-fed’ to them in a way which allows them to grasp the most important information.

In light of the emerging desire for fast, easy-to-understand information, called ‘the bottom-line’ or the ‘top-line,’ respectively, we have arranged BimiLeap to generate its own pair of reports, the first being a PowerPoint comprising all the relevant information (see Figure 7), and then an extensive, formatted Excel file with the relevant information, the summarized tables, and the raw data prepared for additional analysis (see Figure 8).

The report is a presentation-ready PowerPoint. The PowerPoint report is emailed to the researcher immediately after the close of the project, with the typical receipt of approximately one minute (by email). All the information in the report is either boilerplate or dynamically generated from the particular study being analyzed.

Accompanying the PowerPoint report is an excel file, also formatted, and presentation ready. The information page in Figure 8 is only one tab of a multi-tab Excel file, allowing the researcher to do further analysis of the data with other statistical analysis programs.

Part II – Explicating the resulting data – The food Industry and the issue of obesity

Illustrating how a study becomes one block in a scalable knowledge warehouse

An instructive way to appreciate the approach instantiated by the APP is through the data from our history. The study explicated here was presented to a large audience at the 2018 Annual Meeting of the Institute of Food Technologists (IFT), July 2018, in Chicago. The issue of the session was how to communicate as professionals.

The specific topic selected addresses the issue of combating ongoing negative press from activists and others who present their points of view in a variety of formats. The food industry is often accused as active participants in the growth of obesity world-wide. The rationale for such accusations ranges from anger at the R&D efforts which are believed to use any ingredients which make economic sense, to anger at marketing which is accused in the sensationalist press of communicating misleading information to consumers. And then there are those who argue that there is, in some unwritten way, an implicit contract between the company and its consumers for the company to take an active role in the consumer’s health.

How then does the, BimiLeap, work in such a situation, especially in the hands of novices, who do not have the resources of a corporation behind them, a corporation with trained lawyers, a well-paid public relations firm, an advertising (or several) advertising agencies, and a legion of consultants? What happens when the effort must be done expeditiously, to address a problem, and find the necessary messaging? The word ‘expeditiously’ means that there is relatively little time, perhaps a day or two at most, allowing perhaps one to six affordable, rapid iteration, each iteration totally complete, from start to finish in at most two hours. The challenge is to develop this approach, so that in 12 hours one can be fairly certain what to say and do, at least in terms of messaging.

This study, executed rapidly, shows what can be now done (2021), in as short a time as 1-2 hours, using a base of 30 respondents. A base of 100-200 respondents, easy to find on the Internet with online panel providers would take about the same time, approximately 1-2 hours at most. Note that the experiment from the point of view of the respondent takes about 4-5 minutes.

The foregoing sections have shown the set-up of this particular study on obesity and the perceived role of the food industry. We now move to the results, which emerged from the execution of the study with this small number of respondents. Just what emerged? And what conclusions can be made? The reader should keep in mind that the 32 respondents need not be the total number. There could be 132, or more respondents, just as easily. The question is really whether 32 respondents provide enough basic information, and whether one would be better off with the same study comprising 100 respondents, or say three studies, building upon each other, with say 32 respondents each. The answer to the foregoing question is not within the purview of this paper, other than to say that the opinion of the authors is that more studies, not more people for one study, provide a better strategy, assuming of course that there is a requisite minimum number of respondents for any one study.

Data analysis by building a model

The benefit of using the experimental design strategy emerges from the more powerful analytics which deconstruct the combinations, the vignettes, to the contribution of the individual elements, the messages or answers in Table 1. A paragraph is often more realistic than a single element, with the single element providing very little context. The respondent cannot ‘game’ the system, because too many different messages are present. This ‘blooming, buzzing confusion’ forces the respondent to assume a much less intellectualized role, and in turn to respond at almost a ‘gut level,’ the ‘System 1’ response so popular now in research thinking [12].

The actual ratings assigned by the respondent are transformed to a binary scale. The rationale for the binary scale, and thus the rationale for the transformation, is that managers and indeed almost everyone, more readily understands the meaning of a binary no/yes response, versus understanding the ‘meaning’ of a rating scale. Thus, when one is asked to rate ‘feeling about the food industry,’ the answers ‘feel positive’ versus ‘feel negative’ are easier to understand that the graded rating of ‘positive to ‘negative.’ We naturally gravitate to binary response, to no versus yes. For this study, ratings of 1-6 were transformed to 0, ratings of 7-9 were transformed to 100. Afterwards, the program added a very small random number to each transformed value, a value around 10^-5. The random number ensures that the OLS (Ordinary Least-Squares) regression modeling incorporated in BimiLeap system does not ‘crash, when creating the model for the group, or the 32 individual respondent models, even when a respondent limits all the ratings of the 24 vignettes either to the low range of 1-6 (transformed to 0), to the high range (transformed to 100).

BimiLeap also measured the number of seconds from the moment that the test stimulus (combination of elements) appeared on the screen to the time that the response was made. BimiLeap then deconstructed the response time to the contributions of the individual elements, revealing what elements were processed more quickly (shorter response times), and what elements were processed more slowly (longer response times). The response time data warrants a totally separate paper by itself, and thus will not be reported here.

The experimental design and its permutations ensure that OLS regression can build an equation either for each person separately (individual-level model), and a model incorporating the data from many individuals, ranging from the total panel to a group incorporating, for example, only males versus only females (grand models). The specifics of experimental design and regression are well known and accepted in the academic and scientific communities [13].

At either the individual respondent level (so-called individual-level model), or at the group level (so-called grand model, BimiLeap computes the coefficient of a simple regression model, written as: Transformed Rating (Binary + random number) = k₀ +k₁(A1) …. k₁₆D4). The small random number added to the transformed rating ensures that the OLS, ordinary least squares regression, will not crash, even though a particular respondent may have confined all 24 ratings to the range 1-6, or to the range 7-9. Confining the ratings to that range will result in all 24 cases for regression having the same value, 0 or 100, respectively. The small random number makes sure that such a situation will not a happen, for when that situation happens, the OLS regression ‘crashes.

Table 2, specifically the first set of numbers, shows the results from the total panel of 32 respondents. The data in Table 2 are sorted by the coefficient for the total panel. Despite what one might call a ‘thin sample,’ i.e., rather few people, even this small sample shows dramatic differences in the degree to which a specific element can drive a positive versus a negative feeling about the food industry.

Table 2: Parameters of the model relating the presence/absence of the elements to the binary variable ‘Feel positively about the food industry.’ Only positive coefficients are shown for the element.

Table 2 shows the data from total panel, gender, age, and then mind-set segments, to be discussed in the next section. The specific data are not the focus, as much as the structure of the data, and what the data provide.

1. The additive constant tells us the basic likelihood that in the absence of elements, the respondent will have a positive feeling towards the food industry (rating of 7-9 on a 9-point scale, in the absence of any elements). The additive constant is a purely estimate parameter from regression analysis but does have this meaning of a ‘baseline.’
2. Each coefficient tells us the incremental percent of respondents who would be positive (rate the combination 7-9) if the specific phrase, the element or answer, were to be incorporated into the combination. The important coefficients are those 10 or higher. Conventional practice from thousands of these experiments suggests that coefficients around 8 or higher correspond to relevant, meaningful elements in terms of driving external behaviors. The cut-off is not ‘fixed in stone,’ however.
3. Negative coefficients mean that when the element is incorporated into a combination, we would observe a reduction in the number of respondents who would assign a combination the rating of 7-9. The negative coefficients are not relevant to disclose the pattern and are not shown in Table 2. Rather, the cell is left blank.
4. The practical implication here is that one rapid study provides a great deal of specific information need to make a corporation ‘smarter’ in its dealings. The aggregation of these studies creates a database with structured form, linking together the corporation, the topic, the response of different people, all in a coherent fashion with numbers that can be compared to each other.
5. To summarize, the binary variable used as the dependent variable is created by transforming the ratings 7-9 to 100, and the ratings of 1-6 to 0. The rating 7-9 means that the respondent is pro-food industry when reading the vignette. The coefficients and the additive constant can be combined with the sum showing how likely it would be to obtain a rating of 7-9 when the respondent reads the combination comprising the elements whose coefficients are being additive.

Different mind-sets and the opportunity provided through targeted, effective messaging

During the past sixty years, since approximately 1960, marketers and then others have realized that the world does not comprise individuals who think alike. Whereas this statement seems obvious now, it was not always so, especially during the early part of the 20^th century and before, when people were classified by ‘who they were,’ not by ‘what and how they thought.’ Marketers have successfully used the notion of ‘segmentation’ to divide consumers into groups, and have struggled, occasionally successfully, to message to these segments in the appropriate ways [14].

Dividing people into like-groups, segmentation, is often done at first by what people are, and by what they do, and occasionally by how they ‘think’ about a topic. With respect to our project here on the food industry and obesity, the researcher might divide the respondents by who they are (age, gender, body type, where they live, etc.), by what they do (e.g., the types of foods they say they eat, the types of restaurants they frequent, etc.), and by what they think (e.g., how do they respond to topics such as who is responsible for behavior, what is important – health vs pleasure, when it comes to food).

Segmentation of the above type, based on general measures about a person, may or may not help us understand the nature of what is important when it comes to communicating about the food industry and its role (or non-role) in the obesity epidemic. One would hope that dividing the population into traditional subgroups, based either on WHO A PERSON IS, or HOW A PERSON THINKS would reveal clearly different ways of thinking about a topic. Table 2 suggests that dividing people by gender or by age may show differences, not necessarily clear patterns which can form the basis of deeper knowledge and thus more effective actions in the world of messaging. Traditional ways of dividing people assume, without proof, that WHO a person IS determines WHAT a person THINKS. It may not be the case, and in general, it is not the case.

A better way to address the problem of mind-set segments comes from creating these segments from the materials closest to the topic. Beyond the conventional breakout of respondents into self-defined groups, BimiLeap performs a ‘k-means clustering’ [15] of the regression coefficients, first constructing two mind-set segments, then constructing three mind-set segments. These segments are called ‘mind-sets’ because they divide the respondents in the study by the pattern of the coefficients, i.e., by the pattern emerging when the respondent is confronted with a specific, targeted, defined, and quite limited situation.

In our case, we create or perhaps really discover underlying mind-set segments of a specific nature, relating to the issue of how one feels about the food industry, based upon the pattern of responses to the 16 messages. Since each respondent generated a set of 17 numbers (additive constant, 16 coefficients, one per answer), it is straightforward to cluster the respondents based on the pattern of the 16 coefficients.

For these data the k-means clustering, done on the 16 coefficients for the 32 respondents, suggested either two or three different clusters or mind-sets. Selecting the appropriate number of clusters is a subjective matter. The authors’ criteria are parsimony (fewer clusters are better), and interpretability (the clusters should tell a meaningful story).

The two-mind-set solution suggested one group interested in the food industry as a purveyor of ingredients, and the second group interested in the food industry as responsible for the well-being of its customers. Here are the strongest performing elements for the two-cluster solution, i.e., the solution which suggests two mind-sets.

For the purposes of this chapter, it is important to note that the performance of these elements is dramatic, +15 or higher, meaning that dividing people into mind-sets, viz., how they think, produces more knowledge and possibly better actions than simply dividing people by who they are. Such high coefficients almost never occur when these studies divide respondents by who they ARE, for the simple reason that a typical subgroup of individuals who LOOK SIMILAR from the outside often think in many different ways, ways, never be fathomed by simply knowing who a person IS.

Mind-Set 1 of 2 – Focuses on corporate moral responsibility to SELL better ingredients.

B2 Works to improve labels which educate consumers. Shows just what they are buying 23

A1 Tries sincerely to include the best/healthiest ingredients 22

Mind-Set 2 of 2 – Focuses on the corporate responsibility to EDUCATE consumers in healthy eating

D1 Take active responsibility for its customer’s health, and encourages with education and offerings 41

D2 Takes some responsibility for customer’s health, does some reformulation to make offerings more healthful 34

B2 Works to improve labels which educate consumers. Shows just what they are buying 25

D3 Operates fairly, but talks about the need for the customers to take charge..  outreach of education only to customers, schools 25

C1 Supports and funds health-related efforts which educate consumers to eat healthfully 23

The three-mind-set solution suggests that the original Mind-Set 2 of 2 (the food industry is ‘responsible to EDUCATE,’) really comprises two smaller groups, Mind-Set 2 of 3 feeling that the food industry is only partly responsible for the customer’s health/well-being for food, and Mind-Set 3 of 3feeling that the food industry and the customer are partners in the customer’s health-well-being.

Mind-Set 1 of 3 – Focuses on corporate moral responsibility to SELL better ingredients.

A1 Tries sincerely to include the best/healthiest ingredients 29

A3 Uses ingredients with a focus on taste, nutrition, targeting and business 20

B2 Works to improve labels which educate consumers. Shows just what they are buying 19

Mind-Set 2 of 3- Focuses on corporate moral responsibility to EDUCATE consumer.

B2 Works to improve labels which educate consumers. Shows just what they are buying 50

C1 Supports and funds health-related efforts which educate consumers to eat healthfully 32

B1 Since efforts to PREVENT member companies from targeting susceptible populations. Children, obese, low-income 31

D1 Take active responsibility for its customer’s health, and encourages with education  and offerings 25

C2 Partners with government, and cooperates in programs to fight obesity 23

Mind-Set 3 of – Focuses on corporate moral responsibility to SELL better ingredients AND EDUCATE.

D1 Take active responsibility for its customer’s health, and encourages with education  and offerings 44

D2 Takes some responsibility for customer’s health, does some reformulation to make offerings more healthful 38

D3 Operates fairly, but talks about the need for the customers to take charge … outreach of education only to customers, schools 35

The three-cluster solution may or may be useful. Certainly the two-cluster solution helps us a great deal to understand deep differences in the attitudes of the population. Cluster analysis, recall, is simply a heuristic to divide objects, here people, into homogeneous groups, homogeneity defined by mathematical criteria. It is the job of the research to choose the most parsimonious number of mind-sets. The goal is interpretability (the segments must be dramatically different in terms of that to which they respond), and parsimony (the ideal is as few mind-sets as possible.

We have focused on uncovering potentially new-to-the-world mind-sets, simply by running the Mind Genomics experiment. How, then, can the practitioner make use of this discovery? Before we finish up with the last topic, another new discovery, response times (engagement time), let us remain with the discovery of the two mind-sets. The PR strategist working for the food industry, or even a scientist interested in the minds of consumers, might wish to move beyond simply discovering the mind-sets. The practical application of the knowledge tells the food professional in the industry what to emphasize. But what should the professional say to a new person, a person not known beyond basic information about WHO and PURCHASE BEHAVIOR?

Recently, authors Moskowitz and Gere have created a system called the Personal Viewpoint Identifier, or PVI. The objective of the PVI is to create a small questionnaire, six questions long, the pattern of responses to which assign a person to the correct mind-set, with a probability substantially greater than change, although not perfect.

Figure 9 shows the PVI as seen by a respondent, after the PVI is set up. The actual PVI program can be accessed at www.PVI360.com. The research need only put in specified information emerging from the Mind Genomics study, information readily available from the Excel results (see Figure 8). Figure 9 shows three panels.

The left panel acquires relevant information about WHO the respondent IS

The middle panel allows the respondent to answer up to four relevant questions

The right panel presents the six questions in randomized order, these elements selected automatically by the PVI to maximize the chances of a correct assignment.

The PVI is sent out by link, the PVI is done by a person, and the feedback (mind-set membership, feedback information etc.) is sent both to the person and to a database under the researcher’s control. In this fashion, the mind-sets emerging from the study move from ‘nice to know’ facts to a way to optimize messaging to a given individual even immediately after the PVI have been completed.

Discussion & Conclusion

The project of science, and the process empowered by Mind Genomics and BimiLeap

Science, as commonly taught in schools, relies on the isolation and study of variables. There is an often- unstated belief and practice that careful, meticulous, painstaking work is the preferred approach to science. That which we know, according to such worldviews, must be obtained with care, and analyzed with precision. It may be acceptable in the everyday real world to do things ‘in the moment,’ but real science must be done with agonizing precision in order to be perceived as relevant and meaningful.

The research approach promoted in this article moves in a different direction. The notion is that with a reasonably powerful process to acquire knowledge and to understand the world, it may not be necessary to be as meticulous. In fact, the reality of the everyday is that we function quite well by grazing information, rather than sitting down to comprehend the information.

Corporate (and other) learning gained in short, easy-to-do, scalable arrays of studies

The most important outcome of this study is ‘what did we learn,’ and ‘was it worth the effort?’ We can answer those two questions easily. In additional to answering the question for the total panel we discovered new-to-the-world mind-sets with respect to an issue of obesity as a social issue linked with the food industry, Discovering mind-sets in the way we did it, with very few respondents (32), means that we are using the research to uncover sets of ideas which ‘travel together.’ We are not studying the entire world to discover these mind-sets, but rather using a convenience sample to discover the existence of the mind-sets. The distribution of such mind-sets in the population remains an issue to be addressed by larger-scale studies, powered by the PVI, the personal viewpoint identifier.

What is important to keep in mind is that we have established of mind-sets, done with a small sample of randomly chosen individuals, differentiating that discovery from the equal important but quite different question of these segments distribute in the population, another question that we can address, given a parallel piece of easy-to-use software.

There is a second outcome. That is the possibility learning a ‘lot more by simply doing a lot more.’ The evolution of science has been to increasingly larger, more expensive, more ponderous studies. That may be the case for the natural sciences and the physical sciences. As yet, it is not the case for applied science dealing with those topics where ‘the mind of man about external topics is king.’ Where topics can be broken up into aspects, dimensionalized, and studied with small, convenience samples, the Mind Genomics process will flourish, powered by BimiLeap and PV3I60.

Finally, anyone can become a researcher, not just those with advanced degrees, funding, and the permission of an IRB, Internal Review Board. We can envision an entire world of young and not so young researcher, in school, in companies, in play groups, all adding to the sum of knowledge through these experiments.

Envisioning one possible future of knowledge-building made possible by Mind Genomics

The efforts made in this study are, in actuality, quite simple, and the execution expedient. The objective of the effort was not to study the problem in detail, producing an archival document which investigates the problem from many different aspects. This latter approach characterizes much of today’s science, namely the attempt to define the parameters of the problem, and proffer a solution through experimentation, with one solution for one problem added to one solution to another problem, until the general issue is somewhat solved, or at least illuminated by the concatenation of these often-disparate solutions to a set of connected problems.

The approach presented here moves in a different direction, presenting the approach as a standardized method for create knowledge about a topic. There is already the ingoing assumption that the solution will not be perfect, that the range of the aspects studied will be limited, and that the data will be obtained through a small sample of respondents, not a large sample. We can liken this approach to an MRI of the mind, as the mind deals with various topics. Each study, comprising four questions, sixteen answers, perhaps 30-50 respondents, can be likened to one MRI photograph. The brain, or in our case, the mind, does not come into view and cannot be understood by one snapshot, one study alone. Rather, it is the buildup of the snapshots, the dozens, hundreds, thousands, tens of thousands of studies, and perhaps more, which begin to produce a general outline of what’s happening in the mind. Each snapshot alone is a minor effort, but the array of snapshots put together becomes the MRI of the mind, the picture of the mind, taken through this simple, scalable, affordable approach.

Acknowledgment

AG thanks the support of the Premium Postdoctoral Researcher Program

References

1. Moskowitz H, Wren J, Papajorgji P (2020) Mind Genomics and the Law. 1st Edition. LAP LAMBERT Academic Publishing.
2. Parris RR (2009 ) Jury Hands Down Stunning $370 Million Verdict Against Georges Marciano, GUESS? Inc Founder and Art Dealer, Announces R. Rex Parris. GES| 7/28/2009 5:21:29 PM LOS ANGELES, Jul 28, 2009 (Globe Newswire via COMTEX News Network).
3. Moskowitz HR, Porretta S, Silcher M (2005) Concept Research in Food Product Design & Development. Ames, IA: Blackwell Publishing Professional.
4. Luce RD, Tukey JW (1964) Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of Mathematical Psychology 1: 1-27.
5. Anderson NH (2001) Empirical Direction in Design and Analysis. Scientific Psychology Series. Routledge. Taylor & Francis Group New York.
6. Green PE, Krieger AM (1991) Segmenting markets with conjoint measurement. Journal of Marketing 55: 20-31.
7. Green PE, Srinivasan V (1990) Conjoint analysis in marketing: new developments with implications for research and practice. Journal of Marketing 54: 3-19.
8. Moskowitz HR, Gofman A, Itty B, Katz R, Manchaiah M, et al. (2001) Rapid, inexpensive, actionable concept generation and optimization – the use and promise of self-authoring conjoint analysis for the foodservice industry. Foodservice Technology 1: 149-116.
9. Moskowitz HR (2012) ‘Mind Genomics’: The experimental, inductive science of the ordinary, and its application to aspects of food and feeding. Physiology & behavior107: 606-613. [Crossref]
10. Gofman A, Moskowitz HR (2010) Application of isomorphic permuted experimental designs in conjoint measurement. Journal of Sensory Studies 25: 127-145.
11. Moskowitz HR, Gofman A (2004) System and method for performing conjoint measurement. Provisional patent application, 60/538,787, filed.
12. Kahneman D (2011) Thinking, fast and slow. New York: Farrar, Straus and Giroux.
13. Box GEP, Hunter J, Hunter S (1978) Statistics for Experimenters. New York, John Wiley.
14. Claritas (1999) “PRIZM Cluster Snapshots: Getting to Know the 62 Clusters”.
15. Jain AK, (2010) Data clustering: 50 years beyond K-means. Pattern recognition letters 31: 651-666.

DOI: 10.31038/PSYJ.2021341

Abstract

We introduce the emerging science of Mind Genomics to understand how ordinary people feel when they are presented with different vignettes about a couple’s behavior in tough economic times. Respondents each rated 24 unique vignettes describing the economic situation, the time of year, what the couple does in light of coping with the economic situation, and situation at home resulting from the coping efforts. The Mind Genomics method allows the respondent to predict what might happen to the couple. The approach introduces a projective approach to understanding social problems.

Introduction

During a discussion between authors Peer and Moskowitz, the issue arose as to whether there would be a better way to understand the feelings underlying family violence, especially between the adult couple. There are quite a number of papers and books devoted to the topic, so the contribution of this paper is methodological, rather than substantive[1-3].

The literature is filled with different reports about family violence co-varying with economically hard times [4], with the woman taking a job outside the home and the conflicts about the sizes of the salary [5-6], as well as issues such as external factors which would seem unrelated, such as the time of year [7-8] and even external cues such as football games on television [9]. The recent and ongoing Covid-19 epidemic, world-wide, and its seemingly never-end demand on family life is also now an excellent source for family discord, violence, and simply the normal reactions to a drawn-out social stressor.

For the most part, the literature of the family and family issues during time of difficulty approaches the data from the outside in, from observations of behaviors, and attempts to find general patterns. There is a rich world of knowledge from the ‘inside out’, from the point of view of the people in the family, but for the most part this knowledge is confidential, the outcome of private therapy sessions between therapist and family. The topics and issues can be discussed by the therapist in professional meetings and in written form for journals and the like, as long as the relevant identifying information is disguised to accord with privacy laws.

Advancing our understanding by using new tools to quantify, but go from the ‘inside out’

During the past four decades, researchers studying consumer behavior have been interested in questions which move beyond ‘what happened’ or ‘how does the consumer think,’ and into issues that might be called ‘what if thinking.’ The term ‘what if’ refers to the effort to create a model showing what the person might do under different situations. The value of ‘what if’ modeling is patently clear when the issue comes to identifying the decision rules of a person, especially when the objective is to sell the person a product or a service. The objective of all these research techniques is to ‘understand’ the problem [10].

The notion of a model of decision making can move beyond issues of economics, where one might naturally think of the usefulness of the model. What might happen when the modeling is used to create a structure to understand the alternatives possible in everyday behavior, behavior that does not involve a choice among alternatives, but simply a yes/no. For example, what might happen to our knowledge of social issues if we can understand what a person would do in various circumstances?.

In the past decade there has been a concerted effort to understand the mind of people who are presented with description of social situations, instructed to predict of what might happen, using a scale whose numbers are later analyzed to create mathematical models. The research ranges from studies of decision making in courtrooms[11], to studies of social distancing during the time of the Covid-19 epidemic [12], and on to issues involving corruption in the world of education [13]. The approach, Mind Genomics, described below, presents a new approach to understanding how people make decisions, doing so in a way which prevent the respondent from ‘gaming’ the study, giving the answer that the research is expected to hear.

How Mind Genomics works to understand the problem, yet prevent politically correct answers

This paper focuses on a limited topic of home behavior during a period of external economic stress. The approach uses the emerging science of Mind Genomics, a science whose origins can be traced to psychophysics (a branch of experimental psychology), to statistics (specifically experimental design and so-called functional measurement), and finally to consumer research (focus on common, everyday issues, expressed in specifics, rather than in general, and vague language).

Mind Genomics as a science began with the effort to understand how we react to ‘signals,’ or ‘messages’ in the environment. Typically, researchers focusing on the perception and understanding of the external stimuli would identify the test stimuli, and isolate the stimulus and the subject, so that the subject could focus on the stimulus. In this way the researcher could try to eliminate other factors, noise or random variability, which would confound the results. Occasionally, the researcher might wish to introduce distractions as part of the research task, in which case the experiment would be crafted to introduce both a known stimulus and known ‘noise,’ the aforementioned extraneous variability. This approach can be used both for qualitative research (e.g., anthropological research about shopping)[14], or for standard questionnaires.

Mind Genomics went a different direction, deliberating creating combinations of stimuli of known composition (mixtures of messages), presenting these to the respondent and getting an answer, such as a rating. The objective of Mind Genomics is to measure the intuitive response of the subject to the test stimuli, doing so in a noisy environment, but noise which can be factored out during the analysis. In that way, the Mind Genomics effort identifies the subject’s response to the test stimulus, understands the role of the distractor variables, and produces a quantitative measure of the subject’s response. At the same time, it becomes impossible for the subject to ‘game’ the system, viz., to provide so-called politically correct answers of the type that would be socially acceptable, even though misleading.

The Process of Mind Genomics applied to the projection of emotions onto a situation

The study here exemplifies the approach taken by Mind Genomics. In the interests of description, understanding, and discovery, we explain the approach with a case history, one dealing with expected responses in one’s home during a stressful situation. The study was developed from discussions with author Christine Peer. The process follows a set of choreographed steps which move on to a defined experiment generating data that can be immediately analyzed to reveal patterns. In the vernacular of science, one the effort can be defined as a ;cartography,’ to study a social situation, rather than an effort to prove or in contrast to falsify a hypothesis. Mind Genomics, viewed in this context, can be thought of as more ‘description’ of a situation, at least in the mind of a person presented with alternative ideas.

The Mind Genomics process proceeds in a systematic fashion, from the choice of topic and test materials to the creation the test stimuli (vignettes or combinations of messages), the evaluation of the test stimuli, the creation of ‘equations’ or ‘models’ showing how the test stimuli ‘drive’ the responses, and then the extraction of meaning and implications from the data. Over the past six years the process has been templated, allowing anyone to become a researcher (see www.bimileap.com). The templated system, doable even in a demonstration model, sets up the experiment, runs the experiment on the internet, acquires the necessary data, and automatically analyzes the results to generate results usually each to interpret. The statistics are standard ones (experimental design, regression analysis, clustering). The rapid, virtually automatically executed study allows the researcher, even a novice, to spend the valuable time interpreting data, generally data that most people find easy to understand. Patterns emerge clearly, as we will see from these data. With this emerging reality of rapid experimentation, the vision of a science of the mind, a science of the everyday experience, becomes feasible with low cost and little effort, available to all.

Step 1: Select a topic and create the raw materials. The topic sets the focus of the study. Typically, the topic constitutes a circumscribed set of experiences described in words. The topic could be described by a word, or a phrase, portraying a situation. For this study, we were inspired by the opening line of Tolstoy’s novel, Anna Karenina: Happy families are all alike; every unhappy family is unhappy in its own way”. Our topic was the ‘unhappy family.’

Following the choice of topic, create four questions which ‘tell a story.’ The questions are selected to move the ’story along’, but never appear in the test material (viz., the vignette described below). The hardest part of the Mind Genomics exercise often is the selection of the ‘appropriate’ questions because the questions will constitute the backbone of the vignettes, even though the questions never appear. As we see below, the four questions sketch out a reasonable, logical outline. Question B might have preceded Question A, but this decision was to follow the order below.

Question A: What is the current situation of the person?

Question B: What are the local constraints?

Question C: What is the situation of the wife or the husband?

Question D: What happens afterward?

The final part of the first step creates four answers to each question. The answers should be descriptive phrases which ‘tell a story,’ rather than simple yes/no terms which would not be found in a story. The objective is to create small stories, albeit stories without the necessary connectives. The stories or vignettes will comprise 2-4 phrases, presented in centered, stacked format, on a screen.

Table 1 shows the 16 answers, with the answer attached to one of the four questions. It is clear from Table 1 that each of the 16 elements is a simple description, with no hint of the emotional response of the members of the family to each other, although the element can describe the emotional condition of an individual with respect to the circumstances at large, such as elements C3 and C4 about the husband’s emotions in general. It is also clear that the elements paint short ‘word pictures’ and move beyond simply noticeably short and non-evocative phrases. The one element which is noticeably short is B4 (It’s summertime), which is meant to elicit the feelings about summertime.

At this point, it is important to note that the study appears to be simply notions, ideas thought up as convenience stimuli. That is true. The underlying objective of Mind Genomics is to make research easy, quick, iterative, and affordable. Unlike a great number of other research approaches, Mind Genomics encourages guessing about the correct test stimuli to use. Being able to iterate quickly, to pivot in an hour or two, means that the 16 answers or elements shown in Table 1, and indeed even the questions, can be tested in a study, the promising ideas kept and refined, the less than promising ideas discarded, and in their place new ideas introduced.

Table 1: The 16 elements (answers to the four questions)

2. Field execution. The field execution comprises a short, 3-4 minute ‘interview’ with a respondent. The respondent remains totally anonymous, both in terms of disclosure of identity by the panel provider (Luc.id), and by the researcher setting up the experiment. For both Luc.id and the Mind Genomics technology, BimiLeap®, the requirement is for everything to be anonymized, unless specifically requested by the researcher, and accepted by the respondent.

The online panel company, Luc.id, Inc., invites the respondents to participate. The respondents are compensated, but the details of the compensation are not relevant to the researcher. The respondents click on a link embedded in the invitation and are led first to a classification page, which thanks the respondents for participating, and which asks them to record their age, gender, and their marital status (question #3). This third classification question is in the purview of the researcher to write. For this study, the classification question was phrases as: What is your marital situation 1=single2=married3=living together4=divorced5=Not applicable

After the respondent completed the classification question, the respondent next read the orientation page, and rated 24 systematically varied vignettes, using the same rating scale. The BimiLeap program recorded the rating, the response time (time between appearance of the vignette on the screen and the rating), and then recorded the data in a database for almost-immediate analysis.

The respondents are introduced to the study by an orientation and a rating question. By design, the orientation is short. In this way, it is left to the specific elements to specify the situation. It will be the elements which will become the key to understanding the mind of the respondent. The less information in the orientation the more that the respondent will use the elements to drive the rating.

Here is a set of snapshots of families. Please read the full snapshot and tell us what will happen within the foreseeable future. Read the whole snapshot. Is it going to be peaceful, or do you sense some family violence brewing?

What will happen in the foreseeable future with this family?

1=peace and love …9=some violence(Bot=Peace, Top =Violence)

Figure 1: The appearance of a test vignette on the smartphone

Figure 1 shows an example of the vignette as it would appear on a smart phone. The format is set up to make it easy to scan the vignette, pick out the relevant material (an almost automatic behavior), and then rate the combination.

To the unaided eye, the vignettes appear to be haphazard combinations of elements, thrown together at random. The presentation of these types of combinations ‘frustrates’ the respondent who is trying to answer ‘correctly,’ viz., to ‘get it right.’The impression of a haphazard combination is very far from the reality, however, it is important to keep in mind that the 24 vignettes are created according to a strict plan which ensures that each element appears equally, that all elements are statistically independent of each other, and that the data for each respondent suffices to allow for a regression model to be created for that one respondent. The latter feature ensures the ability to estimate the necessary parameters (coefficients), allowing for clustering or segmentation according to the pattern of coefficients.

Each respondent evaluated a specific, and unique set of 24 vignettes, some vignettes comprising two elements (answers from two different questions), some comprising three elements (answers from three different questions), and the remaining comprising four elements (answers from all four questions). Each element appears five times across the 24 vignettes designed for an individual respondent. By unique is meant that the vignettes tested by one respondent were mathematically identical to the vignettes test by other respondents, but the actual combinations were different for each respondent. This approach allows the Mind Genomics effort to cover a great deal of the so-called ‘design space,’ the combinations that could be created. The approach, called the permuted design method[15], makes the Mind Genomics approach a good tool to learn, even when absolutely nothing is known about the topic. One need not know the most ‘promising region’ to test, something which frees the researcher from losing out when the initial guess is incorrect. Each experiment covers a lot of the design space, with as few as 20-30 respondents.

One final point is important to reiterate. This point is the strategy of experimentation which sacrifices precision of measurement (averaging out the variability), replacing it precision by identify the pattern, even though the individual points are variable. The analogy in medicine is the the MRI, magnetic resonance imaging. The pattern emerges from the many different combinations tested. With 50 respondents, for example, the study here covers 1200 combinations. The underlying strategy is to permute these combinations, keeping the mathematical structure the same, but changing the specific combinations [15]. With the 1200 combinations tested in this study (minus a few possible duplicates across respondents), and with measurement at each point, each combination, we have the opportunity to evaluate many different regions of the ‘design space,’ see what works, and then redo the study, focusing on that part of the space. We have the benefit of evaluating many different combinations, and not having to know anything at the start. A few iterations, and a researcher can ‘home in’ to promising areas, viz., topics driving violence.

A total of 50 respondents participated, all from the United States. New subgroups defined as the three mind-sets, will be discussed below. For now it is simply relevant to think of these mind-sets as individuals with different patterns of response to the vignettes.

3. Relating elements to responses. The Mind Genomics point of view is that the valuable information is in the parameters of the model relating the presence/absence of the 16 elements to the ratings. The first step to create the model defines two new dependent variables, both from the rating. Recall that the rating scale is anchored on both side, with 1 being a response of ‘love’ and a 9 being a response of ‘some violence.’ We are interested in the ability of the elements to drive love or violence, respectively. In order to address the issue of two opposite objectives, we create two new binary variables, Love and Violence, respectively. These two newly created binary variables make the interpretation much easier when we look at the tabulated results.

The actual transformation is:

                  Rating of 1-3 transformed to 100 (Love), ratings of 4-9 transformed to 0 (Not love)

                  Ratings 7-9 transformed to 100 (Violence), ratings of 1-6 transformed to 0 (Not violence)

To complete the transformation, a small random number (<10^-4) is added to all of the transformed ratings, in order to add artificial but miniscule variation in the newly created binary variables, Love, Violence. The addition of this small random number ensures that the dependent variable will have some minimal level of variability, required for the OLS (ordinary least-squares regression)o work, and not to crash.

In the presentation of the results, we will presently only the positive coefficients, AND NOT REPORT coefficients which are either 0 or negative, respectively. The underlying themes emerge more clearly when we focus only on the positive coefficients. The negative coefficient simply means ‘absence of’.

4. Relating elements to responses – extending the analysis to individual level models to create new to the world mind-sets. One of the hallmark benefits of the Mind Genomics approach is its ability to uncover mind-sets, defined operationally as different patterns of coefficients for the same set of elements and the same rating attribute. We will create one group of mind-sets, defined a separate, non-overlapping groups of responses who show similar patterns of coefficients, both for Violence and for Love, respectively. That is, we have two types of behaviors, violence (ratings of 7-9) and love (ratings of 1-3). We will create a separate pair of models for each of the 50 respondents, one model or equation for violence vs the 16 elements, and the second for love vs the 16 elements. The independent variables for each respondent were set up by the aforementioned ‘permuted design’, producing a valid experimental design for each respondent. As a consequence, OLS regression allows us to create a valid pairs of equations or models for each respondent.

To create the individual-level equation, we fit a simple linear equation, without an additive constant, doing so for each respondent, once for the 16 elements vs the response ‘violence,’ and for the same 16 elements vs the response ‘love,’ respectively. The calculation generates 32 coefficients, 16 coefficients for the equation for violence, and a parallel 16 coefficients for the equation for love. There are no additive constants in either model.

The clustering which follows is a purely mathematical effort. There is no effort to interpret the data at the tie of clustering, although such an effort might be viable For Mind Genomics studies the clustering produces easy-to-label clusters called mind-sets, easy perhaps because the test stimuli on which the clustering is based, coefficients of elements, use cognitively rich stimuli.

5. Patterns emerging from models. Once the respondents have been identified according to the relevant criteria (total, age, gender, relationship status, membership in the mind-set from clustering) the relevant data for a group are analyzed twice, once creating an equation for Violence (ratings 7-9 converted to 100, otherwise converted to 0), and once creating an equation for Love (ratings 1-3 converted to 100, otherwise converted to 0). This time the equation does have an additive constant.

Binary Variable (Violence or Love) = k₀ + k₁₍A1)+k₂(A2)…k₁₆(D4)

The additive constant is the expected proportion of the responses to be 100 (viz., rating 7-9) when there are no elements. The experimental design introduced above ensures that all vignettes comprised 2-4 elements, meaning that the additive constant is a purely estimated parameter.

Patterns for ‘violence’: The additive constant can be interpreted as the expected percent of the responses the respondent will rate a vignette 7-9 in the absence of elements. Of course, that is not possible since by design all vignettes comprised 2-4 elements. Nonetheless, the additive constant is a valid measure, one that plays the role of a baseline feeling. The top of Table 2 shows the summary table for the response ‘violence’.

1. The total panel is 27 – violence will be the outcome for one out of every four responses
2. Males judge the outcome as violence far more frequently than do females(39 vs 16)
3. Young people judge the outcome as violence more than do older people (31 vs 21)
4. Single people judge the outcome as violence more than people with partners (married, in a relationship)

We now proceed to the individual elements, and the patterns emerging from the groups. As noted above, in the interest of clarity we do not present coefficients which are 0 or negative, but rather present only coefficients equal to or higher than 2. We also shade coefficients of +8 or higher, because it is around +8 that a coefficient reaches statistical significance (T value around 1.5 or higher).

Table 2 (Top; Violence) suggests no clear pattern by key subgroup, but some elements which drive expected violence, at least among some respondent groups. These trigger elements leading to expected violence are:

B1 – Companies are firing employees, as perceived by females and respondents aged 50+. This means that when these respondents read a vignette, the element B1 is likely to trigger the expectation of some violence occurring.

C4 – The husband is sad and depressed, as perceived by females, older, and those with current with partners.

Patterns emerging for ‘love’: Table 2 (Bottom; Love) reveals very low additive constants, most around 12 or lower, except for the younger respondent (age21-49) showing a still-low additive constant of 20.

The two elements bringing almost universal love are descriptions of the season: (B3 – Middle of the winter Christmas) and B4 (It’s summertime)

We conclude from this first analysis that there are few strong differences among the groups. Only a few elements emerge to drive either violence or love.

Table 2: Models relating the presence/absence of the 16 elements to either violence or to love. The data come from the groups as they specified themselves in the up-front classification step.

6. How one element influences another (scenario analysis). The permuted experimental design brings with it an unexpected capability to uncover interactions among elements. The underlying experimental design is set up to make all the 16 elements statistically independent of each other. If every respondent simply evaluated the predesignated 16 combinations, it would be impossible to discover synergies between elements, where the presence of a pair of elements in the same vignette ‘turbocharges’ the rating, so the rating is much higher than one would predict from the simple sum of the coefficients.

The strategy for creating the scenarios follows a set of simple, based upon the notion that each of the elements in the study appears five times for every respondent, and is absent 19 for every respondent. Let us now select one of the four questions, for example question B. Question B comprises four elements presenting information about the time of year, and what companies are doing, respectively. We consider our four elements to be strata, and sort all of the vignettes into the four strata defined by the elements, as well as into the fifth stratum defined by all the vignettes which, by design, lack an element.

The previous exercise creates five strata. In each stratum, the element B is held constant, or does not appear. We now have five new data sets, each with elements from Questions A, C and D present. We simply run two sets of five equations, using as the dependent variables Top3 (Violence) and Bot3 (Love), respectively. The 12 independent variables are A1-A4, C1-C4, and D1-D4.

Table 3 show the five regression models. Each column corresponds to one of the five strata, defined by B1-B4, as well as B0 (B absent from the vignette). Each row corresponds to one of the 12 remaining elements. The top of Table 3 shows the coefficients for Top3 (violence), sorted by ascending order of additive constant. The bottom of Table 3 shows the coefficient of Bot3 (Love), sorted once again by ascending order of the additive constant.

Table 3 shows us much great performance of the elements as drivers of violence and love, respectively. Once the elements are constrained to be fixed in a vignette, they set the ‘stage’ for the ideas. We can see various new patterns emerge, allowing a deeper insight into the topic. For example, when we look at the model for B3 (it’s in the middle of winter Christmas), we see a low proclivity for violence (additive constant is 10, the lowest basic proclivity). ON the other hand, there are specific events which occur which substantially increase the likelihood of violence. Examples are A1 (The local economy is stressed and in recession), and A3 (the children are having problems).

Let us compare the violence expected in winter to the violence expected in summer. We now turn to the last column, for element B4 held constant. The additive constant is much large, an extraordinary 44.Yet, there are no other elements which drive expected violence.

We now move to the bottom of Table 3.We see that the same element, B3 (it’s in the middle of winter. Christmas) brings happiness, viz., synergizes with A2 (the local economy is growing).And, when it is summertime, rather than wintertime, element A3 (the children are having problems) bring love to the family, not violence. That is, the same element (A3)can drive violence (winter) or drive love (summer).

It is patterns like these which are the ‘value add’ to a Mind Genomics cartography. We are able to get a sense of new patterns, some of which make intuitive sense, and some which may spur an ‘aha’ moment.

Table 3: Scenario analyses, holding constant each of the four elements (and the no-element) from Question B, and estimating the model using the remaining 12 elements.

7. The allure of mind=sets as organizing principles. Our previous analyses of the data suggested some effects, such as love expected to emerge during two special times, Christmas in the winter and during summer, respectively. One can investigate the literature of the social and psychological sciences, and in doing so discover these disconnected nuggets which intuitively feel as if they are ‘weak signals’ emerging from a deeper, more coherent reality. The problem is that these signals emerge unexpectedly, and do not allow for a deeper investigation without first requiring a hypothesis of just ‘what is happening’.

Mind Genomics circumvents these problems, first by providing a method of clustering based upon a small, tightly defined topic, and then allowing the research to be done efficiently, inexpensively, and in a manner which moves stepwise through the problem in simple and illustrative steps. The clustering method is totally a theoretical, in terms of the meanings of the clusters. The clusters are labelled by which elements score highest and tell a ‘coherent’ story. The method of clustering is known as k-means clustering. The ‘distance’ between people in k-means clustering is known as ‘D’ defined as (1-Pearson R), where the Pearson R is the Pearson linear correlation between each pair of respondents, computed on the 32 coefficients [16].

The clustering performed on the data did not make any assumptions, because none needed to be made. The models were created for each respondent. The two decisions were to combine the models for one individual (violence and love together to extract people similar in both), and then to extract three mind-sets. Two, three, four, and even more mind-sets could have been extracted. The ideal is to work with as few mind-sets as possible (parsimony), but have each mind-set tell its own coherent story (interpretability). The data suggested that two mind-sets may have been the more parsimonious, but the patterns of the coefficients were not clear. Too much information seemed to cross the mind-sets, suggesting the need for a third mind-sets.

The results from the clustering to generate three mind-sets appear in Table 4 Again, we show only the additive constant, and the elements with positive coefficients for each mind-set. From these, we might name the mind-set. The Top of Table 3 shows the results for the response ‘violence’, the bottom of Table 3 shows the results for the response ‘love’. We will present the additive constant, and then piece together a story from the strong performing elements for that mind-sets.

Mind-Set 1 = High violence constant (42), very low love constant (12). Mind-Set 1 is prone to violence when the economy is stressed and in recession, but that is all. Mind-Set 1 is prone to love when things are better, when its summer and winter and when things are going well. Mind-Set 1 is also, however, just as prone to love when people are getting fired. Mind-Set 1 might be called ‘reactive to the outside world, to when, and to what’s happening’

Mind-Set 2 = lower violence constant (24) and very low love constant (15). Mind-Set 2 is probably a person who is ‘depressive but can be cheered up by the season.’

Mind-Set 3 = lowest violence constant (14), lowest love constant (6), strong reactor to the family situation. Mind-Set 3 is most likely to shut off from the family, miss the time, and feel anxious until the wife begins to clearly help out.

The important thing about Table 3 is that the elements which are strongest appear to paint a picture, which makes intuitive sense. Not everything ‘hangs together’ but we are dealing with a small sample of individuals, and the first effort, done in the period two days. The elements can be refined to expand the focus.

Table 4: Models relating the presence/absence of the 16 elements to either violence or to love. The data come from the three mind-sets which emerged from clustering. Only the positive coefficients are shown.

Discussion and conclusion

As we see from the cursory data from 50 respondents, the data provided by Mind Genomics is rich, indeed far richer than one might expect from a method emerging out of consumer research. One of the reasons for the rich information comes from the effort of Mind Genomics to provide a context for each stimulus. Rather than responding to a set of disconnected questions, the respondent evaluates a unique set of 24 vignettes, each of the vignettes more likely to tell a story than a single question would be.

In our study we take many pictures of the family and ask what might be happening for that particular picture or vignette. It is only later that we put together the individual snapshots (responses to the vignettes) into a coherent whole, an action made straightforward by the use of individual-level experimental designs, and permuted experimental designs. Mind Genomics capitalizes on both, identifying pictures from disparate combinations, and covering a lot of the ‘design space’ of possibilities, using the strategy of permuted experimental design.

The study reported here can be considered to be a cartography, an exploration of the ‘territory’ of the topic, rather than an attempt to confirm or falsify hypotheses. Mind Genomics gives us an opportunity to move in a variety directions, in the spirit of exploratory research, mapping the mind of people as they think about stressful situations, or even as they live through the stressful situation. The objective is not to accept or reject a hypothesis about ‘how behavior works’ or ‘how the mind works.’ Rather, the objective is to find repeating patterns of behavior, or stated patterns of thinking, either separate from the situation, or even in the middle of the situation. A good example of the approach can be found in [17], which deals with the types of behavior that teens want from doctors. That type of information is gathered in the same spirit as these data, namely understanding behavior in stressful situations.

The data lend themselves to the systemized creation of knowledge, literally at an industrial scale, across topics, countries, people, and external situations. For example, we might run this same experiment during several seasons of the year, and in several venues with varying economic conditions, as well as with people who are known to be prone to family violence versus people without that history. All of these approaches will end up creating, in rapid pace, an affordable database of the mind of family violence and family affection, a database that can be extended world-wide with very little effort. The patterns and the increased knowledge, and perhaps even many more ‘ah ha’ moments await the research. The approaches were laid down more than two decades ago, but the methodological advance is fresh, and the data continuing to pile up, in well-managed databases which maintain their value year after year because they reveal the nature of the ‘mind’ and ‘mind-sets’ confronted with situations inevitable emerging from the daily life of people world-wide[18-20].

References

1. Ammerman RT, Hersen M (eds) 2000 Case studies in family violence. Springer Science & Business Media.
2. Denzin NK (1984) Toward a phenomenology of domestic, family violence. American Journal of Sociology. 90 : 483-513.
3. Momirov J, Duffy A (2011) Family violence: A Canadian introduction. James Lorimer & Company.
4. Anderberg D, Rainer H, Wadsworth J, Wilson T (2016) Unemployment and domestic violence: Theory and evidence. The Economic Journal126 : 1947-1979.
5. Aizer A (2010) The gender wage gap and domestic violence. American Economic Review100 : 1847-1859.
6. Showalter K (2016) Women’s employment and domestic violence: A review of the literature. Aggression and Violent behavior31 : 37-47.
7. Rotton, J., & Cohn, E. G. 2002. Climate, weather, and crime. In : RB Bechtel, A Churchman (Eds.), Handbook of Environmental Psychology. John Wiley & Sons, Inc. pg : 481-498.
8. Shortlands ND. https://www.shortlands.co.uk/crisis-at-christmas/
9. Card D, Dahl GB (2011) Family violence and football: The effect of unexpected emotional cues on violent behavior. The Quarterly Journal of Economics126 : 103-143.
10. Moskowitz HR, Gofman A (2007) Selling Blue Elephants: How to Make Great Products that People Want Before They Even Know They Want Them. Pearson Education.
11. Moskowitz, Howard, James Wren, Petraq Papajorgji (2020) Mind Genomics and the Law. 1st Edition. LAP LAMBERT Academic Publishing.
12. Moskowitz H, Prendi V, Gere A, Harizi A, Papajorgji P (2021) Mind-sets of worried citizens and the ‘real-world experiment’ of Covid-19: A mind genomics cartography. Edelweiss Applied Science and Technology 41-49.
13. Gere A, Papajorgji P, Moskowitz HR, Milutinovic V (2019) Using a Rule Developing Experimentation Approach to Study Social Problems: The Case of Corruption in Education. International Journal of Political Activism and Engagement6 : 23-48.
14. Mariampolski H (2006) Ethnography for Marketers: A guide to Consumer Immersion. Sage.
15. Gofman A, Moskowitz H (2010) Isomorphic permuted experimental designs and their application in conjoint analysis. Journal of sensory studies25 :127-145.
16. Likas A, Vlassis N, Verbeek JJ (2003) The global k-means clustering algorithm. Pattern Recognition, 36 : 451-461.
17. Gabay G, Moskowitz HR (2015) Mind Genomics: What Professional Conduct Enhances the Emotional Wellbeing of Teens at the Hospital?. Journal of Psychological Abnormalities.
18. Moskowitz HR (2012) ‘Mind Genomics’: The experimental, inductive science of the ordinary, and its application to aspects of food and feeding. Physiology & behavior107 : 606-613.
19. Moskowitz HR, Gofman A, Beckley, J. & Ashman, H., (2006) Founding a new science: Mind genomics. Journal of sensory studies21 : 266-307.
20. Boserup B, McKenney M, Elkbuli A (2020) Alarming trends in US domestic violence during the COVID-19 pandemic. The American Journal of Emergency Medicine38 : 2753-2755.