About Me

My current research interests involve ways for making startups easier by allowing for more adaptable and reliable foundation models. Really, I'm working towards unemployment by way of advanced language modeling.

Those who I have worked closely with

• Aly Lidayan and Alane Suhr: We worked on ABBEL together, which is a project focused on the data efficient training of long context agents, which compress their own context during policy rollouts. Here is some advice so that others don't repeat the mistakes I made.
• Kartik Goyal: He has been my mentor as I explored Natural Language Processing and alternative forms of text generation, and alignment to human values.
  • We looked into diffusion language models for non autoregressive text refinement. Continuous diffusion language models haven't seen the emirical success of discrete diffusion language models, and ultimately there remain limitations, some of which we outlined in my undergraduate thesis
  • For some time, we also looked at greek to english alignment, and in so doing, I reimplemented IBM translation models 1 - 3. This project wasn't self motivated, and I personally struggled to see the application domains, but if you take a look into the repository, you will find the significant methodological experiements we tried. The technial aspects were very interesting to explore!
  • Quiet-STaR, was an exciting early reasoning work, yet we found with our Quiet-STaR replication study that some results were a result of an improper application of temperature during only the policy gradient calculation but not during sampling, causing an increase in logit magnitudes, which although hurting loss, improves performance on their eval sets. Although somewhat tempering enthusiasim in general targets for the training of reasoning, though the investigation, I came to more deeply understand the tradeoffs which continuous vs discrete forms of reasoning have (e.g. in priors and differentiability), which presents exciting future work!
• George Stoica: During my first research project (2022 September - 2023 September) "ZipIt! Merging Models from Different Tasks without Training" he mentored me in research and allowed me to explore complexities of how representations change through different architectures.
• Ayan Gupta and Kunal Sinha: In Cledge, a startup created by Ayan, Kunal and I worked together (2021 September - 2024 July) on the machine learning team making products for students to more easily write essays and assemble applications. Here is the currently(?) semi-functional essay assistant prototype we worked on!
• Rudy Banerjee: With Capsisco health (2021 Summer) we interned virtually, spending work hours in the garage and going up to play video games when the work was done :).