But now Gemini 1.5 can hold something like 750,000 words in memory, with near-perfect recall. I fed it all my published academic work prior to 2022 — over 1,000 pages of PDFs spread across 20 papers and books — and Gemini was able to summarize the themes in my work and quote accurately from among the papers. There were no major hallucinations, only minor errors where it attributed a correct quote to the wrong PDF file, or mixed up the order of two phrases in a document.

I’m contemplating what topic I want to pitch for the upcoming Applied AI Conference this spring, and I think I want to pitch “How to Cope with AI.”

Case in point: this pull quote from Ethan Mollick’s excellent newsletter.

Every organization I’ve worked with in the past decade is going to be significantly impacted, if not rendered outright obsolete, by both increasing context windows and speedier large language models which, when combined, just flat out can do your value proposition but better.

According to the just-published 2020 U.S. Census data, Monowi now had 2 residents, doubling its population.

This came as a surprise to Elsie, who told a local newspaper, “Then someone’s been hiding from me, and there’s nowhere to live but my house.”

It turns out that nobody new had actually moved to Monowi without Elsie realizing. And the census bureau didn’t make a mistake. They intentionally changed the census data, adding one resident.

Today, I learned about the concept of differential privacy.

Addressing technical debt is rarely about making time for large fixes. It’s about setting strong examples for improving code in our daily work. It’s about celebrating the ability to refactor code to make it easier to work with.

I really like the approach the author takes in categorizing the various types of technical debt one might come across when building software.

The part that I found most enlightening was about yearly debt:

Yearly Debt is the kind where after lots of conversations, someone concludes a rewrite is the only solution. Sometimes a rewrite may be the only solution. Sometimes you may have a Ship of Theseus problem on your hands where you need to slowly and methodically replace parts until the system is the same but different.

Sometimes, though, this isn’t really debt. It’s possible that your dilemma is the result of growth or changing markets. In that respect, calling it debt does a disservice to our success, and distracts from solving the problem of growth.

Brilliant. The “debt” metaphor is apt because not all debt is created equally.

If your town grows into a city, you eventually need to take out debt to build out new infrastructure. You might need to add a few lanes to the main bridge that passes through town. You might need to add more parks or theatres or schools to attract more people.

This incurs debt, for sure, but the payoff comes down the road when you now have an attractive city with amenities that help keep the city vibrant and growing.

The same applies to building software. Sometimes, the algorithm that got you here won’t work for the new customer you want to attract. Sometimes, the frameworks you used to build your mobile app are no longer able to support the hot new feature you want to add.

When framed like that, you no longer call these projects “debt”… you call them investments.

Investments are different from the sort of debt you incur from re-landscaping your back yard for the third time in four years.

This album essentially served as the soundtrack of the early days of the Jed Mahonis Group.

Whenever we needed a day to be heads down, this album would be turned on repeat.

Whenever there was a late night push and we needed the extra motivation to get through it, this album was on repeat.

I came across this video describing the inner turmoil that Daft Punk was feeling while making this album, and I couldn’t help but feel the similarities to my present day situation.

I have long considered this album to be in my top 5 favorites of all time, but this YouTube video made me understand and appreciate it a whole lot more. I should see if there are similar videos for my other favorite albums.

File this video under “reasons I love the internet.”

In October 2023, a group of authors from the Center for AI Safety, among others, published Representation Engineering: A Top-Down Approach to AI Transparency. That paper looks at a few methods of doing what they call "Representation Engineering": calculating a "control vector" that can be read from or added to model activations during inference to interpret or control the model's behavior, without prompt engineering or finetuning.

Being Responsible AI Safety and INterpretability researchers (RAISINs), they mostly focused on things like "reading off whether a model is power-seeking" and "adding a happiness vector can make the model act so giddy that it forgets pipe bombs are bad."

But there was a lot they didn't look into outside of the safety stuff. How do control vectors compare to plain old prompt engineering? What happens if you make a control vector for "high on acid"? Or "lazy" and "hardworking? Or "extremely self-aware"? And has the author of this blog post published a PyPI package so you can very easily make your own control vectors in less than sixty seconds? (Yes, I did!)

It’s been a few posts since I got nerdy, but this was a fascinating read and I couldn’t help but share it here (hat tip to the excellent Simon Willison for the initial share!)

The article explores how to improve the way we format data before it gets fed into a model, which then leads to better performance of the models.

You can use this technique to build a more resiliant model that is less prone to jailbreaking and produces more reliable output from a prompt.

Seems like something I should play with myself!

We come into this world craving the presence of others. But a few modern trends—a sprawling built environment, the decline of church, social mobility that moves people away from friends and family—spread us out as adults in a way that invites disconnection. Meanwhile, as an evolutionary hangover from a more dangerous world, we are exquisitely engineered to pay attention to spectacle and catastrophe. But screens have replaced a chunk of our physical-world experience with a digital simulacrum that has enough spectacle and catastrophe to capture hours of our greedy attention. These devices so absorb us that it’s very difficult to engage with them and be present with other people.

The sum result of these trends is that we are both pushed and pulled toward a level of aloneness for which we are dysevolved and emotionally unprepared. Sartre said hell is other people. Perhaps. But the alternative is worse.

Ironically, this article was shared to me by one of the few people I see IRL nearly every week.

Paul, Micah, Nick, and I get together every Monday night and make music. It’s often the highlight of my week.

We get dinner beforehand and talk about the day to day goings on of our lives. Then, we retreat to Paul’s multi-million dollar recording studio ¹ and just noodle around.

We don’t have a set agenda, no prescribed musical style. One of us just starts playing something, and the rest of us join in.

No matter how depressed, anxious, or frustrated I feel walking into Paul’s house, I never leave with those feelings. Getting to spend time with three smart, talented, and caring dudes always leaves me with a filled bucket.²

Find an activity that brings you joy and go do it with other people. And if you don’t know where to find those people, just ask someone. That’s what Paul did, and thanks to him, I’ve now got two new friends and a weekly outlet for building my guitar skills and expressing some creativity.³

Upon returning from her lunch, my boss asked me to prep one more term packet.

“That poor soul,” I thought as I made my way to the HR supply closet to assemble another fateful folder.

I exited the supply closet, packet in hand, and walked over to my boss’s office in which she and the other HR manager were chatting. As was often the case.

Extending my arm, I reached across my manager’s desk to hand her the freshly prepped term kit. Her hands remained still. Folded on top of her desk.

“You can actually keep that one. That one is for you.”

This is cruelty on another level.

The main reason I’m sharing this is the suggestions Lauren makes after her layoff to find yourself outside of your profession.

It’s a big part of how I’ve spent the last two months since I got laid off. It’s really hard to undo not only 12 years of professional conditioning around the notion that “I am my job”, but also the 16 years of schooling before that which trains you to believe that other people will only value you for your profession.

In one of the job interviews I had this week, someone asked me, “how have you been spending the last two months?”

The only answer I could give was the honest one: “I’ve spent it dealing with my anxiety and depression.”

And while I can’t say I’ve beaten that stuff, things are definitely better. I also can say I’ve been enjoying playing with my kids, dating my wife, learning about AI, hanging out with likeminded nerds exploring AI, playing with new web development frameworks, making music in a band, catching up on TV, finding new ways to exercise (kickboxing and HIIT), and exploring philosophy.

It would be nice to have money coming in the door (and it would be doubly nice to have health insurance to help pay for therapy 😂), but I’m extremely grateful for the opportunity to figure out who I am again.

(For the record, my layoff story isn’t that dramatic. Layoffs are never easy for either side of the table, but they certainly don’t need to be made cruel.)

I ended my first blog detailing my job hunt with a request for insights or articles that speak to how AI might force us to define our humanity.

This op-ed in yesterday’s New York Times is exactly what I’ve been looking for.

[…] The big question emerging across so many conversations about A.I. and work: What are our core capabilities as humans?

If we answer that question from a place of fear about what’s left for people in the age of A.I., we can end up conceding a diminished view of human capability. Instead, it’s critical for us all to start from a place that imagines what’s possible for humans in the age of A.I. When you do that, you find yourself focusing quickly on people skills that allow us to collaborate and innovate in ways technology can amplify but never replace.

Herein lies the realization I’ve arrived at over the last two years of experimenting with large language models.

The real winners of large language models will be those who understand how to talk to them like you talk to a human.

Math and stats are two languages that most humans have a hard time understanding. The last few hundred years of advancements in those areas have led us to the creation of a tool which anyone can leverage as long as they know how to ask a good question. The logic/math skills are no longer the career differentiator that they have been since the dawn of the twentieth century.¹

The theory I'm working on looks something like this:

1. LLMs will become an important abstraction away from the complex math
2. With an abstraction like this, we will be able to solve problems like never before
3. We need to work together, utilizing all of our unique strengths, to be able to get the most out of these new abstractions

To illustrate what I mean, take the Python programming language as an example. When you write something in Python, that code is interpreted by something like CPython² , which then is compiled into machine/assembly code, which then gets translated to binary code, which finally results in the thing that gets run on those fancy M3 chips in your brand new Macbook Pro.

Programmers back in the day actually did have to write binary code. Those seem like the absolute dark days to me. It must've taken forever to create punch cards to feed into a system to perform the calculations.

Today, you can spin up a Python function in no time to perform incredibly complex calculations with ease.

LLMs, in many ways, provide us with a similar abstraction on top of our own communication methods as humans.

Just like the skills that were needed to write binary are not entirely gone³, LLMs won’t eliminate jobs; they’ll open up an entirely new way to do the work. The work itself is what we need to reimagine, and the training that will be needed is how we interact with these LLMs.

Fortunately⁴, the training here won’t be heavy on the logical/analytical side; rather, the skills we need will be those that we learn in kindergarten and hone throughout our life: how to pursuade and convince others, how to phrase questions clearly, how to provide enough detail (and the right kind of detail) to get a machine to understand your intent.

Really, this pullquote from the article sums it up beautifully:

Almost anticipating this exact moment a few years ago, Minouche Shafik, who is now the president of Columbia University, said: “In the past, jobs were about muscles. Now they’re about brains, but in the future, they’ll be about the heart.”

In my explorations of “sustainable business”, I'm often wondering what these two words really mean. I've previously shared some ponderings about the meaning of the word “sustainability”, but what about the word business?

It turns out that it is exactly what it sounds like. The word business originates from Northumbria where the old English word “bisignes” meant care, anxiety or occupation. This evolved into “busyness”, meaning a state of being occupied or engaged. In other words, a state of being busy.

This puts a whole new perspective on the term sustainable business and makes it feel like even more of an oxymoron. If sustainability is the ability to sustain something over the long term, then sustainable business would be to stay busy indefinitely.

Is that viable?

And more importantly, is that what we really want?

As always, Tom’s on point with this essay.

I’m working hard to reduce my wants. Sounds a bit like an oxymoron (no pun intended here), but we’ve all been so conditioned to chase after the shiny thing that we hardly ever stop to ask if the shiny thing is worth coveting.

And it’s really hard to not want to go after the shiny new thing. Getting laid off made it easy to not insta-buy a Vision Pro, but the hype leading up to its release sure got me intrigued.

I’m glad I didn’t, in retrospect, because the reviews aren’t exactly lighting the world on fire.

But this is just one of many examples I can give about being bit by the conspicuous consumption bug.

Another thought: nothing drives me more batty about a job than when you need to track your hours.

The hardest part for me is the obligatory feeling to maximize the time you are claiming you worked.

Let’s say I write down that I spent 8 hours building your website. One of those hours included a meeting where we spent half of it talking about how our weekends went. Ethically speaking, is it wrong for me to charge for an hour of that time, or should I actually say I worked 7.5 hours on your website that day?

Of course, writing this down, it feels silly. Everybody writes down 8 hours.

But if everybody does it, then why do we do it? What gain do we get by tracking our hours? Shouldn’t the final output matter more than how much effort went into building the thing? Is time a useful representation of effort?

I dunno… every time I read Tom’s posts, it feels like there should be a better way to orchestrate our economies. It’s probably time we figure out what symphonies we should be playing before we burn our planet to the ground in the name of growth.

I’d like to use “home” as the operative analogy for my own website.

With any analogy, you choose which properties of the subject to apply to the object of comparison, and which to ignore. What I find significant about homes in this context is that they don’t exist primarily for display: rather, they’re designed around the habits and values of their occupants.

Analogously, I want to use my website to order and document my own activity, and to interact with things and people that I care about.

Still, a website and a home are importantly different in that the former is intended for public exposure, whereas the latter is grounded in private life. But maybe we can relate the public nature of websites to a public dimension of homes: hosting visitors.

Typically we don’t show our house guests everything — we keep many things private and clean up before they arrive. Moreover, we’ve made prior decisions about our furniture and decor with future guests in mind. So homes can certainly be curated for the public eye; but crucially, they maintain their function as living spaces.

I find it generative to consider websites as a similar conjunction of public and private activity: by thinking about how visitors will receive the things that I publish, I’m compelled to produce more and refine the things that I make. At the same time, the website remains my space and is subservient to no other end.

The joy I get from tweaking my personal site and sharing links like this to it seems to be the exact same joy that my kids get out of meticulously organizing their playhouses.