Publications
"The Value of Competitor Information: Evidence from a Field Experiment." Forthcoming, Management Science. Online Appendix.
To what extent are firms knowledgeable of available information on key competitor decisions, and how does competitor information change their own strategic choices? These questions are fundamental to understanding how firms compete and make strategic decisions, yet systematic evidence on them remains limited. I designed a field experiment across 3,218 firms in the personal care industry, where firms randomly assigned to treatment received easily accessible information on competitor prices. At baseline, nearly half of treatment firms were unable to specify competitor prices. However, once treatment firms received competitor information, they were more likely to change their prices, aligning their decisions with competitors rather than differentiating from them. These changes were driven by firms that were more misaligned in their price and quality decisions, and treatment firms subsequently observed higher measures of performance. If competitor information was both easily accessible and decision-relevant, why did firms not use this information on their own? Results from a follow-up experiment suggest that their lack of knowledge may have been driven by managerial inattention. These findings highlight the role that attention may play over information access in improving firm decisions, and suggest that the growing availability of competitor data across many markets may lead firms to align their decisions more with their competitors.
"Decision Authority and the Returns to Algorithms” (with Edward L. Glaeser, Andrew Hillis, Scott Duke Kominers, and Michael Luca). Strategic Management Journal 45 (4), April 2024: 619-648.
We evaluate a pilot in an Inspections Department to explore the returns to a pair of algorithms that varied in their sophistication. We find that both algorithms provided substantial prediction gains, suggesting that even simple data may be helpful. However, these gains did not result in improved decisions. Inspectors used their decision authority to often override algorithmic recommendations, partly to consider other organizational objectives without improving outcomes. Interviews with 55 departments find that while many ran similar pilots, all provided considerable decision authority to inspectors, and those with sophisticated pilots transitioned to simpler approaches. These findings suggest that for algorithms to improve managerial decisions, organizations must consider the returns to algorithmic sophistication in each context, and carefully manage how decision authority is allocated and used.
“Which Firms Gain from Digital Advertising?” (with Daisy Dai and Michael Luca). Marketing Science 42(3), May 2023: 429-439.
In recent years, digital advertising has been on the rise, enabling companies to target their advertising using granular data on customers that were not previously available through traditional channels. We design and run a field experiment on a large review platform across 18,294 firms in the restaurant industry to understand which types of businesses gain more from digital advertising. We randomly assign 7,209 restaurants to freely receive the platform’s standard ads package for three months. The scale of the experiment gives us a unique opportunity to assess the heterogeneity in advertising effectiveness across a variety of business attributes. We find that restaurants that receive advertising observe on average a 7%–19% increase in a wide range of purchase intention outcomes, as well as a 5% increase in customer reviews. We find that gains are heterogeneous across firms, with independent and higher-rated businesses observing larger gains, as well as those with more reviews and higher pre-experiment organic traffic.
“Product Quality and Entering through Tying: Experimental Evidence” (with Michael Luca). Management Science 65(2), February 2019: 596-603.
Dominant platform businesses often develop products in adjacent markets to complement their core business. One common approach used to gain traction in these adjacent markets has been to pursue a tying strategy. For example, Microsoft preinstalled Internet Explorer into Windows, and Apple set Apple Maps as the iOS default. Policymakers have raised concerns that dominant platforms may be leveraging their market power to gain traction for lower quality products when they use a tying strategy. In this paper, we empirically explore this question by examining Google’s decision to tie its new reviews product to its search engine. We experimentally vary the reviews displayed above Google’s organic search results to show either exclusively Google reviews (Google’s current tying strategy) or reviews from multiple platforms determined to be the best-performing by Google’s own organic search algorithm. We find that users prefer the version that does not exclude competitor reviews. Furthermore, looking at observational data on user traffic to Yelp from search engines, we find that Google’s exclusion of downstream competitors may have been effective. The share of Yelp’s traffic coming from Google has declined over this period, relative to traffic from Bing and Yahoo (which do not exclude other companies’ reviews), and Google reviews has grown more quickly than Yelp and TripAdvisor during the period in which they excluded these (and other) reviews providers. Overall, these results suggest that tying has the potential to facilitate entry in complementary markets even when the tied product is of worse quality than competitors.
“Nowcasting Gentrification: Using Yelp Data to Quantify Neighborhood Change” (with Edward L. Glaeser and Michael Luca). American Economic Review P&P 108 (May 2018): 77–82.
Data from digital platforms have the potential to improve our understanding of gentrification, both by predicting gentrification and by characterizing the local economy of gentrifying neighborhoods. To explore, we identify gentrifying neighborhoods using government data, and then use Yelp data to analyze local business activity. We find that gentrifying neighborhoods tend to have growing numbers of local groceries, cafes, restaurants, and bars, with little evidence of crowd-out of other types of businesses. Moreover, local economic activity, as measured by Yelp data, is a leading indicator for housing price changes and can help to predict which neighborhoods are gentrifying.
“Nowcasting the Local Economy: Using Yelp Data to Measure Economic Activity” (with Edward L. Glaeser and Michael Luca). Big Data for 21st Century Economic Statistics (NBER Vol. 79). University of Chicago Press, February 2022.
Can new data sources from online platforms help to measure local economic activity? Government datasets from agencies such as the U.S. Census Bureau provide the standard measures of local economic activity at the local level. However, these statistics typically appear only after multi-year lags, and the public-facing versions are aggregated to the county or ZIP code level. In contrast, crowdsourced data from online platforms such as Yelp are often contemporaneous and geographically finer than official government statistics. In this paper, we present evidence that Yelp data can complement government surveys by measuring economic activity in close to real time, at a granular level, and at almost any geographic scale. Changes in the number of businesses and restaurants reviewed on Yelp can predict changes in the number of overall establishments and restaurants in County Business Patterns. An algorithm using contemporaneous and lagged Yelp data can explain 29.2 percent of the residual variance after accounting for lagged CBP data, in a testing sample not used to generate the algorithm. The algorithm is more accurate for denser, wealthier, and more educated ZIP codes.
Working papers
"Artificial Intelligence and Strategic Decision-making: Evidence from Entrepreneurs and Investors" (with Felipe Csaszar and Harsh Ketkar). Sep 2024. Accepted at Strategy Science.
This paper explores how artificial intelligence (AI) may impact the strategic decision-making (SDM) process in firms. We illustrate how AI could augment existing SDM tools and provide empirical evidence from a leading accelerator program and a startup competition that current Large Language Models (LLMs) can generate and evaluate strategies at a level comparable to entrepreneurs and investors. We then examine implications for key cognitive processes underlying SDM—search, representation, and aggregation. Our analysis suggests AI has the potential to enhance the speed, quality, and scale of strategic analysis, while also enabling new approaches like virtual strategy simulations. However, the ultimate impact on firm performance will depend on competitive dynamics as AI capabilities progress. We propose a framework connecting AI use in SDM to firm outcomes and discuss how AI may reshape sources of competitive advantage. We conclude by considering how AI could both support and challenge core tenets of the theory-based view of strategy. Overall, our work maps out an emerging research frontier at the intersection of AI and strategy.
"The Impact of Communicating Multiple Goals." January 2023. 2nd R&R at Strategic Management Journal.
Firms often pursue and communicate multiple goals, although theory highlights the challenges of doing so. I explore the impact of communicating multiple goals on employee performance, and propose that this effect depends on how far employees are from the productivity frontier. I design a field experiment across frontline employees in a multinational energy corporation as they are evaluated on a procedure, randomly varying (1) whether they are communicated a single goal of safety or multiple goals of safety and efficiency; (2) whether they are additionally provided with an observation of best practices to improve and move closer to the frontier. I find that communicating multiple goals Pareto-improves performance, and observe evidence consistent with the interpretation that this effect is driven by employees inside the frontier.
"Machine Predictions and Causal Explanations: Evidence from a Field Experiment" (with Xi Kang). Aug 2024. Under review.
A key role that decision-makers play in organizations is to provide causal explanations for decisions. Yet despite growing evidence on how machine predictions impact decisions, there has been less insight on how they affect decision-makers' explanations and how this relates to their impact on decisions. We explore this question in this paper, using a field experiment we designed and ran across mutual fund analysts in a leading investment research firm. We randomly assigned analysts to receive predictions for fund ratings based on a proprietary machine learning algorithm developed within the company, and observed their decisions and causal explanations for those decisions over 6 months. We find that even when predictions improve decisions, they can worsen the reasoning of the causal explanations that decision-makers provide for their decisions. Inexperienced analysts working on simpler decisions are especially affected, leading them to provide worse explanations when working with machine predictions. In contrast, machine predictions appear to help improve explanations for more complex decisions, even when they do not improve decisions. Additionally, we find little evidence that providing explainability for machine predictions improves either decisions or causal explanations.
"Real-time Information and Organizational Performance" (with Miguel Espinosa and Sergio Prada). July 2024. Under review.
Recent technological advancements have increasingly enabled data-driven decision-making across firms. While prior literature highlights the value of using more data in decision-making, there has been less insight on the impact of information speed. We examine how information speed influences organizational decision-making, leveraging data from a healthcare context. We analyze the effects of a technology that increased the speed of information by delivering real-time notifications of test results across 64,152 decisions made by 387 physicians. We find that faster information not only expedites decisions but also enhances their quality, resulting in improved organizational performance. These improvements stem from enabling decision-makers to acquire and learn from information more effectively. Thus, our findings indicate that investing in information speed can provide significant advantages from faster and better decisions.
“From Problems to Solutions in Strategic Decision-making: The Effects of Generative AI on Problem Formulation” (with Nety Wu and Chengyi Lin). Under review.
Recent studies demonstrate that large language models (LLMs) can augment ideation and evaluation in strategic decision-making, but their impact on problem formulation remains underexplored. Through a randomized controlled trial with 305 MBA students, we investigate how integrating LLMs into different stages of the decision-making process affects strategic decisions. We find that LLM assistance broadens the scope of problem formulation and increases the number of alternative solutions generated. However, using LLMs in both problem formulation and ideation decreases strategic focus, an effect not observed when LLMs assist only in ideation. We propose a cognitive perspective to explain these results, emphasizing that LLMs may influence the formation and evolution of strategic problem representations by leading decision-makers to reduce cognitive engagement with strategic decisions. Our study advances the understanding of human-AI collaboration in strategic contexts, highlighting the importance of AI integration timing in decision-making.
"Discovering Alternative Strategies: Experimental Evidence on the Impact of Frameworks" (with Nety Wu). July 2024. Under review.
How can decision-makers generate better strategic alternatives? Using randomized experiments across 567 MBA students and executives, we investigate how frameworks influence the generation of strategic alternatives. Our findings suggest that frameworks significantly influence the set of options visible to decision-makers and the ultimate choice they make. Participants who were randomly provided with a framework on strategic options were more likely to generate and select strategic and mutually exclusive alternatives rather than operational improvements, and to explore new alternatives rather than continuing with the status quo. Interview and survey data suggest that frameworks appear to influence how participants generate alternatives by shaping how they formulate the problem at hand. Treatment effects are more muted with more general frameworks, suggesting that the intended objective of the framework matters. Additionally, experiments using GPT-4 suggest that large language models, when prompted with frameworks, can generate more strategic alternatives, indicating their potential role in strategic decision-making.
"Learning from Data in Entrepreneurial Experimentation." August 2024. Chapter in Bayesian Entrepreneurship, edited by Ajay Agrawal, Arnaldo Camuffo, Alfonso Gambardella, Joshua Gans, Scott Stern, and Erin Scott
Experimentation is central for entrepreneurial and strategic decision-making, and involves hypothesis generation, testing, and choice. While much research has focused on hypothesis generation and choice selection, the process of testing --- the “experiment” itself --- remains a black box, with limited insights on how entrepreneurs can design experiments to generate informative data and translate this data into learning. This paper unpacks this process and highlights the challenges entrepreneurs face in learning from experiments. It emphasizes that experiments may often be uninformative, and depends on key choices that entrepreneurs make in designing their experiments. Moreover, it underscores that even informative experiments may not result in learning due to problems of attention and biased updating. Building on these observations, I propose that the value of experimentation is critically dependent on how entrepreneurs generate, attend to, and interpret data from their experiments. Finally, the paper explores how entrepreneurial experimentation may evolve with advancements in artificial intelligence, and proposes that the ability to design informative experiments and make meaningful inferences may become an increasingly scarce and valuable resource.
“Aligning Employee Effort to Strategic Change: The Role of Managerial Gift Exchange” (with Michael Norton). November 2019.
A large literature across strategy and organizational theory suggests that aligning employee behaviors to execute strategic change is difficult. We explore whether gifting can help organizations align employee behavior to strategic change. We propose that gifts may be useful in eliciting behavior reorientation in settings like strategic change where formal incentives may be insufficient, as they are based on reciprocity rather than contingent on measurable outcomes. We examine this empirically by running a field experiment in a company that embarked on a new strategic direction requiring employees to shift from maximizing individual performance to collaborating together. To explore whether and how gifts can help firms align employee efforts to new goals, we introduce in-kind gifts from managers in two ways: one using a relational frame (“Thanks for your hard work collaborating!”) and the other using an additive incentive frame (“Thanks for your hard work collaborating! As a reminder, under the new HR reform, higher team performance will increase your bonus.”). Compared to employees who receive no gift, both gift treatments increase employees' reported willingness to help co-employees by some 30 minutes per day. We also find that the incentive frame significantly reduces the willingness to help of high-performing employees, providing suggestive evidence that gift exchange may work through developing relational ties that motivate desired employee behaviors.
“Measuring Gentrification: Using Yelp Data to Quantify Neighborhood Change” (with Edward L. Glaeser and Michael Luca). NBER Working Paper Series, No. 24952.
We demonstrate that data from digital platforms such as Yelp have the potential to improve our understanding of gentrification, both by providing data in close to real time (i.e. nowcasting and forecasting) and by providing additional context about how the local economy is changing. Combining Yelp and Census data, we find that gentrification, as measured by changes in the educational, age, and racial composition within a ZIP code, is strongly associated with increases in the numbers of grocery stores, cafes, restaurants, and bars, with little evidence of crowd-out of other categories of businesses. We also find that changes in the local business landscape is a leading indicator of housing price changes, and that the entry of Starbucks (and coffee shops more generally) into a neighborhood predicts gentrification. Each additional Starbucks that enters a zip code is associated with a 0.5% increase in housing prices.