Lyle Ungar

• Katherine L. Milkman, Linnea Gandhi, Mitesh Patel, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Jonathan E. Bogard, Ilana Brody, Christopher F. Chabris, Edward Chang, Gretchen B. Chapman, Jennifer E. Dannals, Noah J. Goldstein, Amir Goren, Hal E. Hershfield, Alexander Hirsch, Jillian Hmurovic, Samantha Horn, Dean Karlan, Ariella Kristal, Cait Lamberton, M. Meyer, Allison H. Oakes, Maurice Schweitzer, Maheen Shermohammed, Joachim H. Talloen, Caleb Warren, Ashley Whillans, Kuldeep N. Yadav, Julian J. Zlatev, Ron Berman, Chalanda N. Evans, Rahul Ladhania, Jens Ludwig, Nina Mazar, Sendhil Mullainathan, Christopher K. Snider, Jann Spiess, Eli Tsukayama, Lyle Ungar, Christophe Van den Bulte, Kevin Volpp, Angela Duckworth (2022), A 680,000-Person Megastudy of Nudges to Encourage Vaccination in Pharmacies, Proceedings of the National Academy of Sciences, 119 (6). 10.1073/pnas.211512611 Abstract

  Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was “waiting for you.” Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

• Katherine L. Milkman, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Predrag Pandiloski, Yeji Park, Aneesh Rai, Max Bazerman, John Beshears, Lauri Bonacorsi, Colin Camerer, Edward Chang, Gretchen B. Chapman, Robert Cialdini, Hengchen Dai, Lauren Eskreis-Winkler, Ayelet Fishbach, James J. Gross, Samantha Horn, Alexa Hubbard, Steven J. Jones, Dean Karlan, Tim Kautz, Erika Kirgios, Joowon Klusowski, Ariella Kristal, Rahul Ladhania, George Loewenstein, Jens Ludwig, Barbara Mellers, Sendhil Mullainathan, Silvia Saccardo, Jann Spiess, Gaurav Suri, Joachim H. Talloen, Jamie Taxer, Yaacov Trope, Lyle Ungar, Kevin Volpp, Ashley Whillans, Jonathan Zinman, Angela Duckworth (2021), Megastudies Improve the Impact of Applied Behavioural Science, , 600 (), pp. 478-483. Abstract

  Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

• Pavel Atanasov, Lyle Ungar, Barbara Mellers, Philip Tetlock (2020), Small steps to accuracy: Incremental belief updaters are better forecasters, Organizational Behavior and Human Decision Processes.
• Gilmer Valdes, Albert J. Chang, Yannet Interian, Kenton Owens, Shane T. Jensen, Lyle Ungar, Adam Cunha, Timothy D. Solberg, I-Chow Hsu (2018), Salvage HDR Brachytherapy: Multiple Hypothesis Testing Versus Machine Learning Analysis, International Journal of Radiation Oncology - Biology - Physics, 101 (3), pp. 694-703.
• Barbara Mellers, Eric Stone, Terry Murray, Angela Minster, Nick Rohrbaugh, Michael Bishop, Eva Chen, Joshua Baker, Yuan Hou, Michael Horowitz, Lyle Ungar, Philip Tetlock (2015), Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions, Perspectives on Psychological Science.
• Jonathan Baron, Barbara Mellers, Philip Tetlock, Eric Stone, Lyle Ungar (2015), Two Reasons to Make Aggregated Probability Forecasts More Extreme, Decision Analysis.
• Barbara Mellers, Eric Stone, Pavel Atanasov, Nick Rohrbaugh, S. E. Metz, Lyle Ungar, Michael Bishop, Michael Horowitz (2015), The Psychology of Intelligence Analysis: Drivers of Prediction Accuracy in World Politics, Journal of Experimental Psychology: Applied.
• Barbara Mellers, Lyle Ungar, Jonathan Baron, Jaime Ramos, Burcu Gurcay, Katrina Fincher, Sydney Scott, Don Moore, Pavel Atanasov, Samuel Swift, Terry Murray, Eric Stone, Philip Tetlock (2014), Psychological Strategies for Winning a Geopolitical Forecasting Tournament, Psychological Science.
• Barbara Mellers, Lyle Ungar, Jonathan Baron, Jaime Ramos, Burcu Gurcay, Katrina Fincher, Sydney Scott, Don Moore, Pavel Atanasov, Samuel Swift, Terry Murray, Eric Stone, Philip Tetlock (2014), Psychological Strategies for Winning Geopolitical Forecasting Tournaments, Psychological Science.
• Shay Cohen, Karl Stratos, Michael Collins, Dean P. Foster, Lyle Ungar (2013), Experiments with Spectral Learning of Latent-Variable PCFGs, NAACL-HLT 2013. Related
  Links
• All Research from Lyle Ungar »

Current Courses

• CIS7000 - Conversations And Conversational Bots

  One time course offerings of special interest. Equivalent to a CIS 5XX level course.

  CIS7000003

• CIS8990 - Doctoral Independent Study

  For doctoral students studying a specific advanced subject area in computer and information science. The Independent Study may involve coursework, presentations, and formally gradable work comparable to that in a CIS 5000 or 6000 level course. The Independent Study may also be used by doctoral students to explore research options with faculty, prior to determining a thesis topic. Students should discuss with the faculty supervisor the scope of the Independent Study, expectations, work involved, etc. The Independent Study should not be used for ongoing research towards a thesis, for which the CIS 9990 designation should be used.

  CIS8990042

• CIS9990 - Thesis/dissertation Research

  For students pursuing advanced research to fulfill PhD dissertation requirements.

  CIS9990042

Past Courses

• AMCS5999 - Independent Study

  Independent Study allows students to pursue academic interests not available in regularly offered courses. Students must consult with their academic advisor to formulate a project directly related to the student’s research interests. All independent study courses are subject to the approval of the AMCS Graduate Group Chair.

• AMCS9999 - Ind Study & Research

  Study under the direction of a faculty member.

• CIS0099 - Ugrad Resrch/Ind Study

  An opportunity for the student to become closely associated with a professor (1) in a research effort to develop research skills and techniques and/or (2) to develop a program of independent in-depth study in a subject area in which the professor and student have a common interest. The challenge of the task undertaken must be consistent with the student's academic level. To register for this course, the student must submit a detailed proposal, signed by the independent study supervisor, to the SEAS Office of Academic Programs (111 Towne) no later than the end of the "add" period. Prerequisite: A maximum of 2 c.u. of CIS 0099 may be applied toward the B.A.S. or B.S.E. degree requirements.

• CIS1400 - Intro Cognitive Science

  How do minds work? This course surveys a wide range of answers to this question from disciplines ranging from philosophy to neuroscience. The course devotes special attention to the use of simple computational and mathematical models. Topics include perception, learning, memory, decision making, emotion and consciousness. The course shows how the different views from the parent disciplines interact and identifies some common themes among the theories that have been proposed. The course pays particular attention to the distinctive role of computation in such theories and provides an introduction to some of the main directions of current research in the field. It is a requirement for the BA in Cognitive Science, the BAS in Computer and Cognitive Science, and the minor in Cognitive Science, and it is recommended for students taking the dual degree in Computer and Cognitive Science.

• CIS5200 - Machine Learning

  This course covers the foundations of statistical machine learning. The focus is on probabilistic and statistical methods for prediction and clustering in high dimensions. Topics covered include linear and logistic regression, SVMs, PCA and dimensionality reduction, EM and HMMs, and deep learning. Elementary probability, calculus, and linear algebra. Basic programming experience.

• CIS5220 - Deep Learning/Data Sci

  Deep learning techniques now touch on data systems of all varieties. Sometimes, deep learning is a product; sometimes, deep learning optimizes a pipeline; sometimes, deep learning provides critical insights; sometimes, deep learning sheds light on neuroscience or vice versa. The purpose of this course is to deconstruct the hype by teaching deep learning theories, models, skills, and applications that are useful for applications.

• CIS5450 - Big Data Analytics

  In the new era of big data, we are increasingly faced with the challenges of processing vast volumes of data. Given the limits of individual machines (compute power, memory, bandwidth), increasingly the solution is to process the data in parallel on many machines. This course focuses on the fundamentals of scaling computation to handle common data analytics tasks. You will learn about basic tasks in collecting, wrangling, and structuring data; programming models for performing certain kinds of computation in a scalable way across many compute nodes; common approaches to converting algorithms to such programming models; standard toolkits for data analysis consisting of a wide variety of primitives; and popular distributed frameworks for analytics tasks such as filtering, graph analysis, clustering, and classification. Recommended: broad familiarity with probability and statistics, as well as programming in Python. Additional background in statistics, data analysis (e.g., in Matlab or R), and machine learning is helpful (example : ESE 5420).

• CIS5970 - Master's Thesis Research

  For students working on an advanced research leading to the completion of a Master's thesis.

• CIS5990 - Master's Indep Study

  For master's students studying a specific advanced subject area in computer and information science. Involves coursework and class presentations. A CIS 5990 course unit will invariably include formally gradable work comparable to that in a CIS 500-level course. Students should discuss with the faculty supervisor the scope of the Independent Study, expectations, work involved, etc.

• CIS6200 - Adv Top in Mach Learning

  This course covers a variety of advanced topics in machine learning, such as the following: statistical learning theory (statistical consistency properties of surrogate loss minimizing algorithms); approximate inference in probabilistic graphical models (variational inference methods and sampling-based inference methods); structured prediction (algorithms and theory for supervised learning problems involving complex/structured labels); and online learning in complex/structured domains. The precise topics covered may vary from year to year based on student interest and developments in the field.

• CIS7000 - Cis-Topics

  One time course offerings of special interest. Equivalent to a CIS 5XX level course.

• CIS8950 - Teaching Practicum

  Enrollment for students participating in Teaching Practicum.

• CIS8990 - PhD Independent Study

  For doctoral students studying a specific advanced subject area in computer and information science. The Independent Study may involve coursework, presentations, and formally gradable work comparable to that in a CIS 5000 or 6000 level course. The Independent Study may also be used by doctoral students to explore research options with faculty, prior to determining a thesis topic. Students should discuss with the faculty supervisor the scope of the Independent Study, expectations, work involved, etc. The Independent Study should not be used for ongoing research towards a thesis, for which the CIS 9990 designation should be used.

• CIS9950 - Dissertation

  For Ph.D. candidates working exclusively on their dissertation research, having completed enrollment for a total of ten semesters (fall and spring). There is no credit or grade for CIS 9950.

• CIS9990 - Thesis/Diss Research

  For students pursuing advanced research to fulfill PhD dissertation requirements.

• COGS1001 - Intro Cognitive Science

  How do minds work? This course surveys a wide range of answers to this question from disciplines ranging from philosophy to neuroscience. The course devotes special attention to the use of simple computational and mathematical models. Topics include perception, learning, memory, decision making, emotion and consciousness. The course shows how the different views from the parent disciplines interact and identifies some common themes among the theories that have been proposed. The course pays particular attention to the distinctive role of computation in such theories and provides an introduction to some of the main directions of current research in the field. It is a requirement for the BA in Cognitive Science, the BAS in Computer and Cognitive Science, and the minor in Cognitive Science, and it is recommended for students taking the dual degree in Computer and Cognitive Science.

• COGS3998 - Senior Thesis

  This course is a directed study intended for cognitive science majors who have been admitted to the cognitive science honors program. Upon admission into the program, students may register for this course under the direction of their thesis supervisor.

• COGS3999 - Independent Study

  Departmental permission required

• EAS8980 - PhD CPT

  PhD Student Curricular Practical Training (CPT) credit. Graduate students in Engineering who meet the USCIS eligibility criteria may apply for academic credit for the purposes of F-1 curricular practical training (CPT). In order to be eligible for CPT, students must have already completed one academic year (September to May) of course work, full-time at Penn, but have not completed all of their degree requirements. https://grad.seas.upenn.edu/student-handbook/academic-options/curricular-practical-training/

• ESE8990 - PhD Independent Study

  For students who are studying a specific advanced subject area in electrical engineering. Students must submit a proposal outlining and detailing the study area, along with the faculty supervisor's consent, to the graduate group chair for approval. A maximum of 1 c.u. of ESE 8990 may be applied toward the MSE degree requirements. A maximum of 2 c.u.'s of ESE 8990 may be applied toward the Ph.D. degree requirements.

• GCB9950 - Dissertation

  Ph.D. students enroll in this course after passing their candidacy exam. They work on their dissertation full-time under the guidance of their dissertation supervisor and other members of their dissertation committee.

• LING1005 - Intro Cognitive Science

  How do minds work? This course surveys a wide range of answers to this question from disciplines ranging from philosophy to neuroscience. The course devotes special attention to the use of simple computational and mathematical models. Topics include perception, learning, memory, decision making, emotion and consciousness. The course shows how the different views from the parent disciplines interact and identifies some common themes among the theories that have been proposed. The course pays particular attention to the distinctive role of computation in such theories and provides an introduction to some of the main directions of current research in the field. It is a requirement for the BA in Cognitive Science, the BAS in Computer and Cognitive Science, and the minor in Cognitive Science, and it is recommended for students taking the dual degree in Computer and Cognitive Science.

• PHIL1840 - Intro Cognitive Science

  How do minds work? This course surveys a wide range of answers to this question from disciplines ranging from philosophy to neuroscience. The course devotes special attention to the use of simple computational and mathematical models. Topics include perception, learning, memory, decision making, emotion and consciousness. The course shows how the different views from the parent disciplines interact and identifies some common themes among the theories that have been proposed. The course pays particular attention to the distinctive role of computation in such theories and provides an introduction to some of the main directions of current research in the field. It is a requirement for the BA in Cognitive Science, the BAS in Computer and Cognitive Science, and the minor in Cognitive Science, and it is recommended for students taking the dual degree in Computer and Cognitive Science.

• PSYC1333 - Intro Cognitive Science

  How do minds work? This course surveys a wide range of answers to this question from disciplines ranging from philosophy to neuroscience. The course devotes special attention to the use of simple computational and mathematical models. Topics include perception, learning, memory, decision making, emotion and consciousness. The course shows how the different views from the parent disciplines interact and identifies some common themes among the theories that have been proposed. The course pays particular attention to the distinctive role of computation in such theories and provides an introduction to some of the main directions of current research in the field. It is a requirement for the BA in Cognitive Science, the BAS in Computer and Cognitive Science, and the minor in Cognitive Science, and it is recommended for students taking the dual degree in Computer and Cognitive Science.

• PSYC4998 - Mentored Research

  Mentored research involving data collection. Students do independent empirical work under the supervision of a faculty member, leading to a written paper. Normally taken in the junior or senior year.

• PSYC4999 - Honors Mentored Research

  The Honors Program has been developed to recognize excellence in psychology among Penn undergraduates and to enhance skills related to psychological research. The 4998 credit signifies an Honors Independent Study, completed as part of the Honors Program. The honors program involves: (a) completing a year-long empirical research project in your senior year under the supervision of a faculty member (for a letter grade). This earns 2 cu's. (b) completing a second term of statistics (for a letter grade) before graduation. (c) participating in the year-long Senior Honors seminar (for a letter grade). This seminar is designed especially for Psychology Honors majors; this receives a total of 1 cu. (d) participating in the Undergraduate Psychology Research Fair in the Spring semester, at which honors students present a poster and give a 15-minute talk about their research. (e) a total of 15 cu's in psychology is required. Students will be selected to be part of the Honors Program in the Spring of their junior year (see application process online)

• STAT9950 - Dissertation

  Dissertation

Knowledge @ Wharton

• Overwhelmed by Too Many Choices When Shopping? Wharton Retail Expert Weighs In, Knowledge @ Wharton - 12/19/2024
• Michael Lee, Oatly North America’s EVP & Executive Creative Director, Knowledge @ Wharton - 12/19/2024
• Why Anchor Institutions Are Vital to Local Economies, Knowledge @ Wharton - 12/17/2024
• Is the American Dream of Homeownership Still Within Reach?, Knowledge @ Wharton - 12/17/2024
• Even Under Trump, AI Governance Is Here to Stay, Knowledge @ Wharton - 12/17/2024
• 2024 Ripple Rewind (Part 2), Knowledge @ Wharton - 12/17/2024
• What Are the Best Countries of 2024?, Knowledge @ Wharton - 12/16/2024
• Your Pride Could Affect Your Workplace Reputation, Knowledge @ Wharton - 12/12/2024
• Samantha Baker, VP of Commerce at theSkimm, Knowledge @ Wharton - 12/12/2024
• NFL Stats & Analysis with Ben Solak, Knowledge @ Wharton - 12/11/2024