Will training data matter anymore for self-driving cars?

The coolest thing about these new LLMs is their ability to handle few-shot learning. Give it a few examples, and GPT-3 will extrapolate that out to whatever else you throw its way. There’s no need for hundreds of thousands of pieces of training data just to classify a paragraph’s sentiment as positive” or negative”.

It makes me think a generally-intelligent system capable of few-shot learning will replace almost all neural nets trained on insanely big data sets.

Take driving. All humans in the world can learn how to drive in under an hour with almost 0 training miles.1 Put any teenager behind the wheel, show them the gas pedal and then how the brakes work. Tell them which side of a road to stay on. Tell them not to hit other cars. Stop at red lights. Otherwise, go.

And off they go, with shockingly few issues.

Tesla Autopilot has now been trained on what, a trillion miles driven? And they’re still having issues with roundabouts? I’ve watched tons of those Cruise videos too. While they’re insanely impressive, they also seem to suffer from brittle edge cases.

Self-driving neural nets seem to need training data for every single possible driving scenario in order to properly inch into traffic, take turns, and stop for pedestrians. Sort of like how IBM Blue needed to ingest every chess game ever played in order to take down Garry Kasparov, Tesla autopilot is insatiable in its thirst for training data. And even though it’s drowning in data, Autopilot gets nervous and stuck all the time.

Will a trillion more training miles really help much at this point?

Prediction: the first real self-driving system will be trained on less than 100 miles of driving.

Instead of being fed a billion or a trillion miles, we’ll simply show the system the rules of the road and off it will go - just like how GPT-3 only requires 1-2 examples in order to accurately perform classification, completion, and data extraction.

Billions and trillions of miles trained not required.


  1. I’ve previously written this about our narrow band of intelligence:

    We’ve spent over three decades, millions of man-hours, and tens of billions of dollars trying to teach computers how to intelligently stay between the lines.

    Yet, a few years before Alan Turing built the first computer, my grandfather was on a tomato field in rural Virginia. In two weeks, out of necessity, he figured out how to slip the red stickshift tractor-trailer into first gear, and then back to neutral. Into first gear again, and then back to neutral. Then all the way up to third gear and into town, to haul the tomatoes off. He was 11 years old.

    Nearly any human who has tried to learn how to drive has been able to do so in a short amount of time. Over 70 years since Turing’s first machine, we still don’t have self-driving cars.

    ↩︎

Two things have changed since 1990

I was born in 1990. Two things have changed since then:

  1. Water fountains
  2. Lightbulbs

A couple years ago, I landed in Albuquerque en route to Taos. The jet bridge from the Southwest plane to the terminal was a time machine to the 1990’s. Every store logo had that brutalist Seinfeld font aesthetic. The wallpaper was a blue-splash pattern like those old coffee cups, the windows were small, the brick was multicolored, and there wasn’t a water fountain in sight.

It took me back to elementary school, standing in line after gym, waiting for my turn at the dinky metal drink fountain. I’d push my entire body weight against the panel hoping for just a dribble of water.

Now all of those dinky fountains have add-ons sitting on top, which fill bottles at a torrential pace. And everybody seems to carry a bottle everywhere they go. When did this bottle craze begin?

The only change bigger than the water bottle has been the lightbulb. I dropped a plastic IKEA LED bulb yesterday. It hit the floor. Nothing happened.

I still remember my dad slicing his hand as he tried to catch a falling glass bulb. He probably juggled it because it was insanely hot after taking it out of its socket. The radiant heat from those bulbs made reading in the summer a tortuous event.

I do miss the startling pop” you’d hear every once in a while, when the little wire inside the bulb would burn to a crisp and leave a black burn mark on the glass.

There are lots of things I don’t remember. I don’t remember NFL games being grainy. I don’t remember how all the movie trailers had that corny deep voice explaining In A World…”. I don’t remember packages taking a long time to ship. I don’t remember wanting to listen to a song, but not being able to.

Everything seems the same in retrospect. Everything except the fact that humans were camels, and all homes were lit with a warm yellow glow.

What’s in and out for 2023

I made this list on New Year’s Eve. So far, so good.

In

  • 1pm-9pm eating window
  • Bill’s Pizza Night every week
  • Daily writing
  • AI
  • Sunday tea
  • Fewer todos
  • Short sprints instead of long jogs

Out

  • Scrolling in bed
  • Scrolling in general
  • Fake deadlines
  • Deli meat

My experience Wednesday in SF

It’s pouring rain.

I take the Bart to Fidi. Everybody on the train is wearing a mask. As we roll to a stop at Embarcadero, the train loses power. We wait for the generator to come on so we can open the doors.”

Finally outside, I pass 10 people on the three-block walk to 345 California. Half are our unhoused friends.”

The Cafe-X coffee robot tent at the corner of Pine is no longer there. Somebody is bundled up, sleeping in its place.

I roll into Industrious at 9:15am. All of the bagels from the breakfast spread are gone. This isn’t San Francisco’s fault! Although since most private offices are empty, I’m not sure how I missed the rush.

I get some third wave drip coffee and begin my morning scroll.

I learn Facebook is laying off 13% of their employees. Just down Market Street, Twitter is embroiled in an internal war - payroll, servers, and debt payments all competing over declining ad revenue.

Worst of all, it turns out the guy with the curly hair plastered on every billboard around town is a total scam who rug pulled every FTX customer, creating an $8 billion hole in his balance sheet and sending crypto into a free fall. Bitcoin is down to $14k, not far off its price when I left San Francisco in August of 2020.

I take a Lyft to dinner. We get stuck behind two ambulances on Polk. As we idle, watching a woman get loaded onto a stretcher, I realize nine years ago I was picked up for my first-ever Lyft ride just a block away. I tell my driver about this personal history, about how magical it felt to order a car at the push of a button, how a Honda Accord adorned with a pink mustache pulled up to the curb, how I donated the suggested $8 to be driven to a bar near AT&T Oracle Park.

He asks me how much Lyft is charging me for this ride. $22.50.

He grunts. They quoted me $9.

I get to the bar where I’m meeting two of our investors. One has just come from a CEO roundtable. He decides not to drink because he has to wake up early now.” Everybody is planning layoffs, he explains. People are burning millions a month. Capital markets don’t exist. I don’t think half of these people are going to have businesses in a year.

A couple hours later, I schlep back to Oakland. Lyft quotes me $0.50 more than Uber, so I flip a mental coin and go for the Uber. Boy does my gamble pay off. I’m matched with a Model 3.

In bed by 12:30, awake four hours later, and hungover from my three glasses of wine, it’s time to catch my Delta Main Cabin 5.5 hour flight back to NYC.

I forgot my laptop

I got home and unzipped my backpack. My laptop wasn’t in its padded pocket.

I fired up Find My… and there it was, pulsing on the map, at Park and 17th about 1.5 miles away.

My first reaction was to panic. How am I going to provision these new accounts?

And then my second reaction was calmer. Maybe I don’t need to do anything tonight.

This is the first night I’ve slept further than 10 meters away from my laptop since our 2019 trip into the Kyrgyzstan backcountry.

So here I am, writing.

It’s nice knowing I can’t do the things I have the urge to do tonight. I was going to manually provision accounts for new Bottle 2.0 merchants — but I shouldn’t still be doing those anyway. Instead, we need to build an internal New Merchant form.

My dad jokes that in college his job was to tap the keg. For four years he tapped every keg. When I graduate, surely they’ll be living in a beer desert, he thought.

August rolled around and nobody called him. The keg got tapped. People partied.

My forgotten laptop is an act of forced delegation. I don’t have access to my console. All I can do is write a spec for an internal form, and then wait for it to get built.

No git checkout -b billy/fixes for me tonight.

… I do still have my iPad though :)

My Kindle is dead

I ordered a paperback book from Amazon last night. The last book I ordered was a paperback, too. So was the one before that.

In every dimension, the Kindle is better. It’s slimmer, lighter, holds thousands of books, syncs to the computer, has a built-in dictionary, simplifies highlighting, and I can read it on red-eyes without turning on that glaring overhead light.

Yet there’s something about seeing The Wind-Up Bird Chronicle on the bedside table that nudges me to pick it up. There’s something different about turning the page, about feeling how much more’s left to read. Real books hit different.

I keep seeing surveys where workers say Work From Home is better than being in the office in every way. The commute is shorter, there’s time for deep work, it’s cheaper, it means more time spent with family and healthier lunches and fewer wasted meetings and less time spent getting dressed.

Last month, Andy went back into Industrious for the first time in a year. He called me as he walked home. It’s amazing what we humans justify. I thought WFH has been fine, but after a day in the office, I feel like we’ve lost a year of our lives.”1

Yes, going in to work sucks. Everybody’s cramming on to the train at the same time. The bougie lunch stall is hawking $15 sandwiches. It’s a drag to shave and put on pants and talk about how yeah this rain is crazy” 30 times. Being in-person is worse in every single dimension.

But dammit, I love running to catch that train. That’s my favorite lunch spot. And sometimes you’ve got to keep shaving to stay alive.

I keep getting introduced to people in NYC. Not a single person has suggested we do a Zoom call.

Let’s grab coffee this week.”


  1. I am long remote work and flexible work. WFH != remote work. Bottle is a remote company. I’ve been bouncing around for the past three years. But the death of the office is vastly exaggerated.↩︎

Page 1 of 29 Older →