I've had this idea of an ideal research agent for at least 4 months now, but a ton has gotten in the way: (a) a new baby, (b) Transpose Health work, and (c) that in the world of computer science, I (half-jokingly) sit in the "valley of despair" of the Dunning-Kruger Effect: I am competent enough to write my own pretty-good code, aware enough to know that I am slower and less efficient than full-time developers, but stubbornly proud enough to not have adopted AI-driven coding.
Seeing my ideas flounder for almost half a year, undeveloped, made me rethink this misplaced pride. (That, and my core competencies are thinking of new ideas to try out (not necessarily writing the code for it!); hearing Andrew Ng say that any good coder is using AI; wanting to test out "vibe-coding" to build out apps; real-life, time-saving use cases in my time on the AI/Q Fund at Connecticut Innovations; and Google releasing their Antigravity IDE.)
I include a journal of my progress below, some pre-coding thoughts, and a summary of the Antigravity pilot.
Some side thoughts:
- Why release an IDE now? Software development is one of the few areas that has shown very clear acceptance of AI agents (see: Andrew Ng on the 20VC podcast above), and with a great IDE, Google can both (a) win market share quickly, (b) collect more data on software code, and (c) create lock-in for its Gemini LLM (in a market where I believe LLMs will become a commodity). The future of software development is slowly becoming clearer: human developers, enhanced by AI. If we compare software development to building a house, AI agents today are high-quality pre-fab cabinets. It saves the builder time without sacrificing quality, but doesn't replace the need to have a builder to begin with.
- AI is lowering the barrier to entry for creating high-quality apps. I guess this shouldn't be surprising. In the internet age, drag-and-drop sites like Wix and Squarespace let you build no-code websites, and so no-code automation and AI-driven app development shouldn't be a surprise.
- There are so many frameworks out there for everything. Part of the challenge of building anything today is that there are so, so many libraries (in Python) that do variants of the same thing, and it's hard to know which to use. Each library requires its own small learning curve, and each never works exactly like I want it to. The house-building comparison: the aisles and aisles of hammers, table saws, and lumber you find at Home Depot.
- Compliance/security still remains a very high barrier to entry. I've been investigating building a health records SaaS platform, which would require HIPAA-compliance and SOC-2 Type II compliance. I'd imagine Antigravity can build the software to meet these requirements ... but would it be able to help direct me with the non-software requirements for these certs? That'd be incredible.
Antigravity workflow thoughts:
- Overloaded servers. I had to keep restarting the agent to keep coding. Everyone must be piloting this at the same time!
- Great product/market fit for someone like me. I have enough coding background to be picky about how I like things coded/abstracted, but not enough to be efficient at coding it myself. I can be a picky a
Takeaways after a day of Antigravity coding:
- The value of investment memos is thinking and unearthing risks. This is at the crux of what I'm trying to build here: not an investment memo generator (a passive tool), but ideally a research co-pilot, automating the dull stuff and pushing you to think more.
- Communication is key. AI-assisted coding shifts the art of coding to the art of clearly communicating specs and business needs -- which definitely requires a different skill set. (Oftentimes, when I code, I'll "wander" and see what's possible, reshaping the specs as I go.)
- AI coding (like Antigravity) foretells the future of work elsewhere. I was surprised that in some ways, AI coding was more like a dialogue between me and the AI coder: me telling it what to do, and it responding back with its thought process. (A more experienced AI coder may be better equipped to tel lthe AI agent exactly what to do.) It almost feels like the type of product that I'm trying to build: the core workflow is the dialogue between human and AI; the memo is just a happy by-product.
- A few distinct tasks ("agents") are needed for a good investment memo. In my initial app, I broadly lumped all tasks together. However, I think there's a few distinct task types that should be abstracted out:
- Research analyst - the analyst "read" a few different articles and websites, pick out pertinent quotes, presents them in the report, and synthesize them together. A virtual hunter/gatherer.
- Quantitative analyst - an agent that can work through a data room spreadsheet and create projections, calculate IRR, etc. Would also be great to do things like calculate betas for public-market comps, P/E ratios for comps, etc.
- Guesstimator analyst - a consultant-y analyst who can build up TAMs from the bottom-up and top-down, based on clearly stated assumptions.
- Devils advocate analyst - comes up with counterfactuals (e.g. the bear case in an industry analysis), helping to uncover risks and good questions to ask companies/founders.
- Legal analyst? - at some point, you probably want an agent who can see how things compare to a standard corpus (e.g. how do the deal terms presented match against the terms we normally agree to)
- Would be nice to separate out AI and personal/human thought. In a perfect next iteration, it'd be nice to clearly separate what's AI-driven content from my human personal thoughts and opinions. I think this will be critical to show where a single person is still adding value.
= = =
Initial prompt
I started with an app to do a market analysis because (a) it's something that standard ChatGPT can do pretty well (example) and (b) it doesn't require any "private" data (like a start-up's slide deck or internal call notes). Here's the initial prompt I gave it:
i'd like your help to develop an agentic website/web app that helps do market research.
GOAL
- the user should initially be able to input a market (e.g. "ai-driven internet browsers") and the system should (a) estimate a TAM, ideally in a few ways (e.g. with a few different starting point assumptions, bottoms-up/top-down), (b) collect public-market comps, and (c) asses the competitive landscape.
- the system should cite sources or assumptions its using, and it shoudl make an unbiased estimate of how trustworthy the source is
- in the future, the user should also be able to input files to help the system do assessment
- the system should also state where it has uncertainty or where more research is needed
- the system shoudl also ask the user questions if it has them (e.g. clarifying the industry, etc.)
- the user should also be able to ask follow-up questions
- the end product should be a collaboratively-driven write-up of the market landscape
LANGUAGE
- im most familiar/comfortable with python, so ideally the output would be rooted in python
Next step: Planning
Antigravity auto-generated an implementation plan. I can leave comments.
Next step: Iterative testing and feedback-giving
Antigravity auto-generates an implementation plan, then ticks tasks off the checklist. I kept tacking new ideas on (e.g. add a chat interface so you can interact and update the memo, allow a user to upload a document, etc.), which muddied the code quite a bit.
There is no magic: the man behind the curtain
The "magic" of AI comes with prompting the LLM the right way. For example, here's the LLM call that happens behind the scenes (you can copy/past this directly into ChatGPT yourself).
Here's the TAM estimate taken from other sources:
external_prompt = f"""
Context:
{context}
Task: Find existing market size estimates for {market} from the sources.
Format:
- List each estimate as: "$X billion/million by YEAR [Source Name]"
- Include key assumptions if mentioned
- Be concise
"""
Here's the consultant-style TAM estimate:
topdown_a_prompt = f"""
Market: {market}
Task: Estimate TAM using a top-down approach.
Use macro-level assumptions (e.g., % of global population, GDP, industry size).
Format:
- Assumption 1: [specific assumption]
- Assumption 2: [specific assumption]
- Calculation: [show math]
- Result: $X billion/million
"""
Final state: A decent draft
I ended up with a decent but not great proof-of-concept after about 4 hours of on-and-off tinkering. The web app runs an internet search, aggregates the results, and generally kind of works. But it's a bit poorly architected right now, in a way that I'm not happy with. My plan is to scrap this project and "rebuild" it from scratch with what I've learned; the good thing is there's no sunk cost fallacy with the existing code.
