Barbara Mellers

• Pavel Atanasov, Jens Witkowski, Barbara Mellers, Philip Tetlock (2024), Crowd prediction systems: Markets, polls, and elite forecasters, International Journal of Forecasting. Abstract

  What systems should we use to elicit and aggregate judgmental forecasts? Who should be asked to make such forecasts? We address these questions by assessing two widely-used crowd prediction systems: prediction markets and prediction polls. Our main test compares a prediction market against team-based prediction polls, using data from a large, multi-year forecasting competition. Each of these two systems uses inputs from either a large, sub-elite or a small, elite crowd. We find that small, elite crowds outperform larger ones, whereas the two systems are statistically tied. In addition to this main research question, we examine two complementary questions. First, we compare two market structures, continuous double auction (CDA) markets and logarithmic market scoring rule (LMSR) markets, and find that the LMSR market produces more accurate forecasts than the CDA market, especially on low-activity questions. Second, given the importance of elite forecasters, we compare the talent-spotting properties of the two systems, and find that markets and polls are equally effective at identifying elite forecasters. Overall, the performance benefits of “superforecasting” hold across systems. Managers should move towards identifying and deploying small, select crowds to maximize forecasting performance.

• Barbara Mellers and Siyuan Yin (2023), Reference-Point Theory: An Account of Individual Differences in Risk Preferences, Perspectives on Psychological Science. https://doi.org/10.1177/17456916231190393 Abstract

  We propose an account of individual differences in risk preferences called “reference-point theory” for choices between sure things and gambles. Like most descriptive theories of risky choice, preferences depend on two drivers—hedonic sensitivities to change and beliefs about risk. But unlike most theories, these drivers are estimated from judged feelings about choice options and gamble outcomes. Furthermore, the reference point is assumed to be the less risky option (i.e., sure thing). Loss aversion (greater impact of negative change than positive change) and pessimism (belief the worst outcome is likelier) predict risk aversion. Gain seeking (greater impact of positive change than negative change and optimism (belief the best outcome is likelier) predict risk seeking. But other combinations of hedonic sensitivities and beliefs are possible, and they also predict risk preferences. Finally, feelings about the reference point predict hedonic sensitivities. When decision makers feel good about the reference point, they are frequently loss averse. When they feel bad about it, they are often gain seeking. Three studies show that feelings about reference points, feelings about options and feelings about outcomes predict risky choice and help explain why individuals differ in their risk preferences.

• Pavel Atanasov, J. Witkowski, Barbara Mellers, Philip Tetlock (Under Review), The person-situation debate revisited: Forecasting skill matters more than elicitation method.
• Philip Tetlock, Lu Yunzi, Barbara Mellers (2022), False Dichotomy Alert: Improving Subjective-Probability Estimates vs. Raising Awareness of Systemic Risk, International Journal of Forecasting.
• E. Karger, J.T Monrad, Barbara Mellers, Philip Tetlock (Under Review), Reciprocal scoring: A method for forecasting unanswerable questions.
• Katherine L. Milkman, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Predrag Pandiloski, Yeji Park, Aneesh Rai, Max Bazerman, John Beshears, Lauri Bonacorsi, Colin Camerer, Edward Chang, Gretchen B. Chapman, Robert Cialdini, Hengchen Dai, Lauren Eskreis-Winkler, Ayelet Fishbach, James J. Gross, Samantha Horn, Alexa Hubbard, Steven J. Jones, Dean Karlan, Tim Kautz, Erika Kirgios, Joowon Klusowski, Ariella Kristal, Rahul Ladhania, George Loewenstein, Jens Ludwig, Barbara Mellers, Sendhil Mullainathan, Silvia Saccardo, Jann Spiess, Gaurav Suri, Joachim H. Talloen, Jamie Taxer, Yaacov Trope, Lyle Ungar, Kevin Volpp, Ashley Whillans, Jonathan Zinman, Angela Duckworth (2021), Megastudies Improve the Impact of Applied Behavioural Science, , 600 (), pp. 478-483. Abstract

  Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

• Ike Silver, Barbara Mellers, Philip Tetlock (2021), Wise teamwork: Collective confidence calibration predicts the effectiveness of group discussion, Journal of Experimental Social Psychology .
• Siyuan Yin, H. Arkes, John McCoy, Morris A. Cohen, Barbara Mellers (2021), Conflicting Goals Influence Physicians’ Expressed Beliefs to Patients and Colleagues, Medical Decision Making, 41 (5), pp. 505-514. https://doi.org/10.1177/0272989X211001841 Abstract

  Background

  Physicians who communicate their prognostic beliefs to patients must balance candor against other competing goals, such as preserving hope, acknowledging the uncertainty of medicine, or motivating patients to follow their treatment regimes.

  Objective

  To explore possible differences between the beliefs physicians report as their own and those they express to patients and colleagues.

  Design

  An online panel of 398 specialists in internal medicine who completed their medical degrees and practiced in the United States provided their estimated diagnostic accuracy and prognostic assessments for a randomly assigned case. In addition, they reported the diagnostic and prognostic assessments they would report to patients and colleagues more generally. Physicians answered questions about how and why their own beliefs differed from their expressed beliefs to patients and colleagues in the specific case and more generally in their practice.

  Results

  When discussing beliefs about prognoses to patients and colleagues, most physicians expressed beliefs that differed from their own beliefs. Physicians were more likely to express greater optimism when talking to patients about poor prognoses than good prognoses. Physicians were also more likely to express greater uncertainty to patients when prognoses were poor than when they were good. The most common reasons for the differences between physicians’ own beliefs and their expressed beliefs were preserving hope and acknowledging the inherent uncertainty of medicine.

  Conclusion

  To balance candor against other communicative goals, physicians tended to express beliefs that were more optimistic and contained greater uncertainty than the beliefs they said were their own, especially in discussions with patients whose prognoses were poor.
• Ville Satopää, Marat Salikhov, Philip Tetlock, Barbara Mellers (2021), Decomposing the Effects of Crowd-Wisdom Aggregators: The Bias-Information-Noise (BIN) Model, International Journal of Forecasting.
• Christopher Karvetski, Carolyn Meinel, Daniel Maxwell, Lu Yunzi, Barbara Mellers, Philip Tetlock (2021), Forecasting the Accuracy of Forecasters from Properties of Forecasting Rationales, International Journal of Forecasting.
• All Research from Barbara Mellers »

PSYC 253 Judgments and Decisions

PSYC 600 Proseminar in Judgments and Decisions

MDS 521 Judgments and Decisions

MKTG 211 Consumer Behavior

MKTG 960 Seminar in Consumer Behavior

Current Courses

• PSYC6000 - Judgment & Decisions

  Choice of half or full course units each sem. covering a range of subjects and approaches in academic psychology.

  PSYC6000303

Past Courses

• BDS5210 - Judgments & Decisions

  This course addresses the ideal standards of judging and deciding, and the ways in which people fall short of these standards, with emphasis on the latter. We will discuss heuristics and other intuitive strategies that people may use in day-to-day thinking, and the biases that result from this use. We will apply this approach to shed light on faulty analyses in medicine, law, and everyday thinking. Understanding the ideals of good thinking and causes of our failure to conform to these ideals may ultimately help improve the decisions we make in private and professional lives. Permits offered to non-MBDS students if space is available.

• BDS5990 - Capstone Experience

  The new capstone experience consists of two semesters where students are invited to engage in a dialogue with industry to understand the landscape of the field of Behavioral Sciences, and use the tools learned in MBDS courses to tackle specific industry problems. In the first semester (Fall), students engaged in a conversation with industry leaders that provided them with first-hand knowledge of a) how behavioral science is applied in organizations across several domains, b) why BeSci insights and tools are becoming increasingly relevant in organizational settings, and c) what MBDS graduates can expect when they finally get to the job market. In the Spring, students will participate in professional development workshops as they embark on a Design Challenge, when, in teams, they will have a chance to use instructional and strategic input from these conversations to shape their work. Our goal is to prepare our students to engage with industry and global organizations as soon as they complete the program.

• MKTG2110 - Consumer Behavior

  This course is concerned with how and why people behave as consumers. Its goals are to: (1) provide conceptual understanding of consumer behavior, (2) provide experience in the application of buyer behavior concepts to marketing management decisions and social policy decision-making; and (3) to develop analytical capability in using behavioral research.

• MKTG3990 - Independent Study

• MKTG9500 - Jdgmnt & Dec Making Cb A

  The purpose of this course is to provide a solid foundation for critical thinking and research on the judgment, decision-making and choice aspects of consumer behavior. There is a focus on how people process information when making judgments and choices and how the processes of judgment and choice might be improved. Topics of discussion include rationality, judgment under uncertainty, judgment heuristics and biases, risk taking, dealing with conflicting values, framing effects, prospect theory, inter-temporal choice, preference formation, and the psychology of utility. The focus will be on the individual decision-maker, although the topics will also have some applicability to group and organizational decision-making and behavioral research methodologies.

• MKTG9510 - Judmnt & Dec Making Cb B

  The purpose of this course is to build off MKTG 950, "Judgment and Decision Making Perspectives on Consumer Behavior - Part A" with a more specialized focus that will vary from year to year. This course is intended for those interested in deepening their study of Judgment and Decision Making beyond the basics.

• MKTG9990 - Independent Study

  Requires written permission of instructor and the department graduate adviser.

• PSYC3730 - Judgment & Decision Making Sem

  This course is designed to help you become a better decision maker. By the end of the semester, you should have the skills to approach decision making from a broader perspective with new tools and a new awareness of many common errors and biases. You will learn about normative decisions (how people should make choices if they want to use principles of rationality, logic and probability), descriptive decisions (how people really do make decisions) and prescriptive decisions (how people can make better decisions given normative principles and what we know about human behavior). We’ll discuss the theoretical foundations of the field, some of the key empirical insights. We’ll discuss what it means to have good judgment and how experts and novices differ. We look at decision making in such as public policy, medicine, the law, business, and intelligence analysis. Decision making is something we do every day, many times a day. It is so natural that some people don’t even realize they are doing it. Many of the insights from this field have real-world implications.

• PSYC4998 - Mentored Research

  Mentored research involving data collection. Students do independent empirical work under the supervision of a faculty member, leading to a written paper. Normally taken in the junior or senior year.

• PSYC4999 - Honors Mentored Research

  The Honors Program has been developed to recognize excellence in psychology among Penn undergraduates and to enhance skills related to psychological research. The 4998 credit signifies an Honors Independent Study, completed as part of the Honors Program. The honors program involves: (a) completing a year-long empirical research project in your senior year under the supervision of a faculty member (for a letter grade). This earns 2 cu's. (b) completing a second term of statistics (for a letter grade) before graduation. (c) participating in the year-long Senior Honors seminar (for a letter grade). This seminar is designed especially for Psychology Honors majors; this receives a total of 1 cu. (d) participating in the Undergraduate Psychology Research Fair in the Spring semester, at which honors students present a poster and give a 15-minute talk about their research. (e) a total of 15 cu's in psychology is required. Students will be selected to be part of the Honors Program in the Spring of their junior year (see application process online)

• PSYC6000 - Proseminar in Psych

  Choice of half or full course units each sem. covering a range of subjects and approaches in academic psychology.

• PSYC6999 - Indiv Res for 1st Yr Grd

  Individual Research for First-Year Graduate Students

• PSYC9999 - Independent Study

  Individual Study and Research

Knowledge @ Wharton

• How Dynamic Electricity Pricing Can Improve Market Efficiency, Knowledge @ Wharton - 11/12/2024
• Is Algorithmic Management Too Controlling?, Knowledge @ Wharton - 11/12/2024
• The Hidden Cost of AI Energy Consumption, Knowledge @ Wharton - 11/12/2024
• Charitable and Political Giving | Pinar Yildirim, Knowledge @ Wharton - 11/12/2024
• The Future of Banking, Knowledge @ Wharton - 11/11/2024
• Election Statistics, Knowledge @ Wharton - 11/06/2024
• Robots Are Taking Over Low-skilled Jobs — and Changing Votes, Knowledge @ Wharton - 11/05/2024
• Crypto Politics: Liberals’ and Conservatives’ Confidence in Cryptocurrencies, Knowledge @ Wharton - 11/05/2024
• Leadership and Elections | Mike Useem, Knowledge @ Wharton - 11/05/2024
• Automation Is Harming Low-skilled Workers and Possibly Their Voting Preferences, Knowledge @ Wharton - 11/01/2024