Research
Papers
Sparsity Moves Computation: How FFN Architecture Reshapes Attention in Small Transformers
Author(s): Gabriel Smithline, Chris Mascioli
University of Michigan | Accepted to the ICML 2026 Mechanistic Interpretability Workshop | Preprint arXiv:2605.09403
Architectural choices inside the Transformer feedforward network (FFN) block do not merely affect the block itself; they reshape the computations learned by the rest of the model. We study this effect in one-layer Transformers trained on digit addition with carry, modular arithmetic, and histogram counting. Comparing dense FFNs, gated linear units (GLUs), mixture-of-experts (MoE), and MoE-GLUs, we find that sparse MoE routing can shift computation from FFN to attention, with the strongest ablation-visible effect on carry-based addition. We decompose this redistribution into reduced per-token FFN capacity and sparse partitioning across experts. Critically, frozen random routing nearly matches learned routing, suggesting that redistribution is driven largely by architectural sparsity rather than router-learned specialization. As a secondary finding, GLU-style multiplicative gating rotates task-relevant Fourier structure out of the per-neuron basis and into distributed subspaces, making neuron-level interpretability less informative while preserving structured computation. We validate these conclusions with random-routing, narrow-FFN, and top-2 MoE controls, plus parameter-matching, activation-function, and width-scaling analyses. Together, these results show that local FFN design choices can have nonlocal consequences for Transformer computation.
SGRD: Solver-Guided Rationale Distillation for Finetuning Strategic Reasoning
Author(s): Chris Mascioli, Gabriel Smithline, Michael P. Wellman
University of Michigan | Under review
Large language models (LLMs) for decision making offer advantages in natural language interaction and potential for providing interpretable justifications for their actions. For complex strategic environments, strong play often requires expensive reasoning-time computation, and even then generally cannot match the performance of specialized game-solving algorithms. We introduce Solver-Guided Rationale Distillation (SGRD), a pipeline for distilling solver-aligned actions combined with LLM-generated decision rationales into a cost-effective language-model policy. SGRD samples game states, labels each state with the action generated by a specialized solver, and asks an LLM teacher to justify that action based on the acting player's information. It then fine-tunes a student model to emit a concise rationale followed by a valid action. We instantiate the method in an alternating-offer bargaining game with private valuations and outside offers, using a PPO self-play policy as the solver and gpt-oss-20b as the student. In matched cross-play against the PPO solver over 1000 evaluation seeds, the distilled student achieves mean payoff of 351.2 value-units, representing a +51.8 surplus over its outside option and a 39.2% deal rate, while emitting only ∼188 completion tokens per turn. This is roughly 16× fewer than the same base model run at medium reasoning effort, which it nonetheless improves by +10.9 value units (p < 0.001, paired permutation test). These results suggest that solver-guided rationale supervision can transfer strategic behavior from non-linguistic game-playing agents into language-model policies without requiring expensive teacher-model inference at deployment.
Measuring Competition and Cooperation in LLM Bargaining: An Empirical Meta-Game Analysis
Author(s): Gabriel Smithline, Chris Mascioli, Mithun Chakraborty, and Michael Wellman
University of Michigan | 1st place and winner of the AgentX–AgentBeats Competition, hosted by UC Berkeley RDI | National Bureau of Economic Research Conference on Econometrics and Mathematical Economics 2026 | Early draft presented at ICML 2025 MAS Workshop | [PDF]
We conduct an empirical game-theoretic analysis of large language models (LLMs) negotiating to divide subjectively valued items, as a case study of assessing advanced AI in mixed-motive settings. Bargaining agents built on proprietary LLMs represent meta-strategies, mapping prompts describing the game to negotiation policies. A graduated series of prompting levels steers agents away from blatant negotiation errors. Across a variety of bargaining scenarios, we estimate empirical meta-games over meta-strategies that include LLM agents, three heuristic strategies representing extreme negotiation attitudes, two reinforcement-learning-derived policies, and an agent based on common bargaining heuristics. We evaluate agents at meta-game equilibria in terms of individual effectiveness, social welfare, and fairness, using bootstrap methods to quantify uncertainty. Our analysis reveals a positive association between individual effectiveness and social metrics across models, with performance varying systematically across providers, architectures, and prompting levels. Behavioral analyses further illuminate why certain models excel or fail in specific bargaining regimes and uncover distinct qualitative patterns across OpenAI, Anthropic, and Google models.
Matching at the Midpoint: A Strategic Agent Based Analysis
Author(s): Gabriel Smithline, Anri Gu, and Michael Wellman
University of Michigan | 2025 | ACM International Conference on AI in Finance 2025 (ICAIF) | Selected for Oral Presentation (top 15%) | [PDF]
We study midpoint matching through Nasdaq's Midpoint Extended-Life Order (M-ELO) mechanism, which offers non-displayed midpoint execution subject to a mandatory holding period to balance liquidity and adverse-selection concerns. We conduct an empirical game-theoretic analysis using an agent-based simulation in PyMarketSim that models both M-ELO and a traditional lit order book, simulating traders' venue choices across a range of holding periods and enumerating pure and mixed Nash equilibria for each setting. We analyze welfare outcomes, strategic stability via deviation graphs, and basins of attraction across these equilibria. Our findings reveal that shorter holding periods can insufficiently deter higher frequency small lot trading, while excessively long delays erode the midpoint premium for large traders. This highlights the fundamental trade-off in designing holding periods that effectively screen predatory trading without undermining value for intended users.
Fast and the Furious: Hot Starts in Pursuit-Evasion Games
Author(s): Gabriel Smithline and Scott Nivison
AFRL Research Project | 2024 | Accepted to ARMS (Autonomous Robots and Multirobot Systems) Workshop at AAMAS 2025 and selected for oral presentation | [PDF]
Effectively positioning pursuers in pursuit-evasion games without prior knowledge of evader locations remains a significant challenge. A novel approach that combines game-theoretic control theory with Graph Neural Networks is introduced in this work. By conceptualizing pursuer configurations as strategic arrangements and representing them as graphs, a Graph Characteristic Space is constructed via multi-objective optimization to identify Pareto-optimal configurations. A Graph Convolutional Network (GCN) is trained on these Pareto-optimal graphs to generate strategically effective initial configurations, termed "hot starts". Empirical evaluations demonstrate that the GCN-generated hot starts provide a significant advantage over random configurations. In scenarios considering multiple pursuers and evaders, this method hastens the decline in evader survival rates, reduces pursuer travel distances, and enhances containment, showcasing clear strategic benefits.
Creating a Cooperative AI Policymaking Platform through Open Source Collaboration
Preprint arXiv:2412.06936
Advances in artificial intelligence (AI) present significant risks and opportunities, requiring improved governance to mitigate societal harms and promote equitable benefits. Current incentive structures and regulatory delays may hinder responsible AI development and deployment, particularly in light of the transformative potential of large language models (LLMs). To address these challenges, we propose developing the following three contributions: (1) a large multimodal text and economic-timeseries foundation model that integrates economic and natural language policy data for enhanced forecasting and decision-making, (2) algorithmic mechanisms for eliciting diverse and representative perspectives, enabling the creation of data-driven public policy recommendations, and (3) an AI-driven web platform for supporting transparent, inclusive, and data-driven policymaking.
Quantal Response-Based Defense in One vs. Many Stackelberg Security Games
Tufts University | 2023-2024 | [PDF]
During the 2023-2024 academic year, I researched Stackelberg Security Games and Deception. My work focused on finding reductions from Stackelberg Security Games to social planner problems in macroeconomic models.
Understanding Eviction Rates Induced by COVID: A Statistical, Mechanism Design, and Game Theoretic Perspective
Lafayette College | 2021
During the 2021 academic year, I researched the impacts of COVID-19 on the rental market in Pennsylvania and Easton. I approached this through statistics, econometrics, mechanism design, and game theory. I worked with the city of Easton to help implement these Mechanism Design solutions.
