Startup update 13: Slop, slop, everywhere

Originally posted 2025-07-20

Obligatory disclaimer: all opinions are mine and not of my employer

Progress Update

After having powered through the weekend to try and finish the AP chemistry course release, I’m exhausted and declaring defeat. It’s like 99% done, but I’m going to take some days off and will wrap up when I get back.

I spent some time introspecting about why it’s been a month since my last public release. While I’ve been very productive this last month and working on what I believe to be useful things, I’m not actually any closer to PMF, and I don’t have traction or a group of parents/students eager to hear about this latest work.

Mistake 1: too ambitious. I decided a month ago that I’d build a full-blown AP Chemistry course, thinking that it probably wouldn’t take that long. Instead, I should have picked something a lot less ambitious - say, a single lesson on redox reactions or a unit on electrochemistry. That would let me test 1. the efficacy of a scripted lesson, delivered via LLM and 2. the efficacy of my problem generator + solution grader. Even simpler, I could have dribbled out a series of medium-hard difficulty problems on Facebook/Tiktok or something and encourage students to interactively solve the problems via my guided problem solving harness on these problems.

Mistake 2: I picked the wrong data model. I modeled the course as an organized set of lesson/problem templates, and when the user clicked “enroll in course”, these templates would be cloned to create a whole set of lesson conversations and problem solving conversations. This meant that once a user enrolled, they would get a version of the course that was frozen in time at the time it was cloned! Knowing this, I kept on polishing and pushing the launch date, knowing that students would not see fixes to the course if I made them after release. Instead, I should have cloned the template as late as possible - so that if I edit a lesson script or add a new recommended problem to a topic - the student will always be able to go back and see the latest version of the course. This would enable me to publish a half-completed course and complete the rest over time (if there was demand).

A third thing that slowed me down, but was less of a foreseeable mistake: I thought I’d save time by compiling the AP chemistry teacher’s manual. But instead, the manual turned out to be a bad source to compile, and I ended up having to rewrite a good chunk of the course anyway. On top of that, my Unit/Topic/Lesson data model was basically dictated by the manual. As a result, I had lots of extraneous fields, fields at the wrong level of abstraction (e.g. “Extra Resources”, or “Excluded Topics”, or “Key Equations”, “Suggested Skills”, “Learning Objectives” and other useless crap), incorrect granularity of topics, etc. etc. I was constantly trying to make incremental edits to the database schema, and trying to keep the whole thing aligned, making each iteration slower. Claude then noticed all these extraneous details and kept on adding junk at every level of my app - course compiler, backend features, frontend features, which continued to slow me down as I waded through extra unnecessary code. Instead, I should have iterated on a single lesson, then topic, then unit, just to hammer out the details of the data model at each level.

To fix these mistakes, I ended up basically rewriting the entire problem backend from scratch, from the ground up, with the benefit of hindsight and all the accumulated experience of the last month. The data model is stupidly simple, compared to what it started off as: A course has many units, which each have many lessons, each of which has a lesson script and a set of recommended problems. It took 2 days to hammer out everything and build out all the FastAPI endpoints and new UI to visualize/interact with it.

Slop, slop, everywhere

While I was test-solving a bunch of my problems, I dumped a few of them into Perplexity, thinking, surely it should be able to just spit out some balanced reactions without much difficulty, right? Instead, I inadvertently stumbled upon an AI-generated minefield of completely synthetic questions and synthetic answers that are completely incorrect. This website has been indexed by Perplexity, and Perplexity was just cribbing bad answers from it.

I was also looking for a quick and easy feedback widget that would let users submit feedback/complaints/whatever to me. A Perplexity search for “UI feedback widget” turned up a bunch of promising results, but then I realized in the sources, that literally everybody is following the AI-SEO playbook by writing useless listicles that promote their own solution as the best. Isn’t there any independent developer who’s actually reviewing or commenting on these things?

Well, then, I thought I’d ask Reddit, and I thought I was getting some interesting hits in this thread and then I realized that literally every single suggested app in the comments section is an astroturf - if you look at the comment history of any of those users, you can see that they are obviously the developer of that app, or a marketing person. So at the end of the day I decided I’m just going to vibecode a feedback widget.

I can’t trust any of this shit anymore. I want my 2000s-era internet back. I will seriously pay for an extension that highlights and/or removes slop from the internet.