Andy Matuschak
patreon
Towards scalable blip cultivation
🕑 Added 2024-11-01 00:28:19 +0000 UTCI’ve been collaborating with my friend Taylor Rogalski on an experimental new programmable attention system. Our work is still quite early, so in lieu of screenshots and prototypes, I’d like to share some of our high-level conceptual design thinking.
The problem
My life is filled with neglected blips of salience. They need my attention to thrive, but they don’t fit into a to-do list or my main creative process. So, mostly, they get lost. A few examples:
Questions: I re-watched Bret Victor’s “Inventing on Principle” and wondered: insofar as I’m inventing “on principle”, what are those principles?
References: On a stroll, Gena Gorlin suggested that psychological research on the “stages of change theory” might be relevant to my work on timeful texts.
Insights: While journaling, I noticed my implicit belief that great work needs righteous conviction and grand vision. But my friend Rob Ochshorn has a playful, loose attitude and yet consistently inspires me. Maybe it’s safe to be more playful.
Ideas: Chatting with Matthew Siu about use cases for Rewind-style continuous screen recorders, an idea emerged: a button to show the recorded moment before any browser tab was opened, so you can see “where it came from”.
Quests: I’ve stumbled a few times on drawabox.com, a website for learning to draw. Each time I think: this looks cool, but also enormous and overwhelming. I don’t immediately know how to engage.
Yearnings: I want some regular venue to sing harmony with talented people, but I don’t want to perform gigs or maintain a repertoire—just singing new music together, each time.
I know that if I put these blips in my to-do list, they’re probably just going to rot. Maybe I could move these blips along if I gave them my deep creative attention, but that limited resource is usually committed to more coherent projects.
Part of the trouble is that I often don’t know how much I care about these blips. A quickly-jotted scrap like “look into stages of change theory” doesn’t (yet) mean much to me, so I’m hesitant to give it deep attention. But I don’t understand my relationship to this blip enough to just drop it, either. It might become important, if I engaged a bit. Meanwhile, it’s trapped in a frustrating zone of ambiguity.
Complicating matters, many of these blips aren’t really “completable” in a single sitting, no matter how focused. For example, consider “what are my principles?”, or “maybe it’s safe to be playful” or “try drawabox.com”. These call for repeated engagement over time. But it’s tough to effectively orchestrate my attention that way.
In summary, blips are hard to handle because they often:
lack clear next steps
require creative brainpower
may or may not be important to me
aren’t relevant now, but may be in some future context
can’t be addressed in one sitting
I have some attentional systems that work well: I know how to do big, focused creative projects; I know how to maintain routinized habits and interests. But where do those things come from? My sense is that they often grow out of the blips we’ve been discussing—an idea, a question, a reference, a yearning. But it’s remarkably haphazard. When the blip arrives with enough clarity, or coincides with a stretch of free time, or repeats itself many times, it can grow into something real. Otherwise, it ends up haphazardly dropped, but not because I decided to drop it. All this leaves me with a sense of loss.
One obvious parallel here is to spaced repetition memory systems, which address the haphazard nature of remembering. If I add something to my memory system, it’s no longer up to chance. I can be quite confident that I’ll remember it. Likewise, for well-specified operational tasks, Getting Things Done-style systems (usually) give me confidence in place of what feels like chance.
So: how might we construct a working attentional system for these blips?
Our approach
We propose:
a feed which resurfaces blips over time
with lightweight activities to move them forward
in low-energy Twitter-scrolling moments on your phone
until you have enough clarity to “do” the blips, or let them go
I’ll illustrate with an example. Suppose I quickly capture “look into stages of change theory”. Then:
The next day, I see that blip in my feed with a task suggesting I set a timer for two minutes and ramble into the microphone about how this came up, and why it might be interesting. My speech is transcribed, cleaned up, and saved with the blip.
A couple days later, that blip resurfaces with a summary of the theory from Wikipedia and links to a few articles. I read the summary and an abstract. Then I note that I’m particularly interested in the interventions, and how they might apply to spaced-repetition systems that support self-help books.
A few days later, that blip resurfaces with a list of popular interventions. It suggests that I mark any that seem particularly relevant.
Over the next few weeks, during idle periods, I read little relevant bits of papers describing those interventions, ramble observations into the microphone, and slowly form a high-level understanding of how this theory might fit into my work.
Finally, I build enough clarity that I feel comfortable letting go. I tell the blip to snooze for six months, or until I mention “timeful texts” in another blip.
It might seem strange that this example ends with inaction. But this is actually a very real success. In idle moments of time across a few weeks, I came to understand enough about this theory to see how it might fit into my work, and to draw connections in the future. I accumulated that understanding in notes I can use later. I know that I’ll see this blip again if it’s likely to be relevant, so I feel safe letting it go. It’s not dead: it may well turn into a key part of a future project. By contrast, if I’d added this blip to my to-do list, I doubt any of this would have happened.
Let’s see how the system addressed our earlier list of problems with blips. We said they:
lack clear next steps
The system proposed some next steps.
require creative brainpower
The activities are tuned for low energy periods and require only brief spurts of creative energy.
may or may not be important to me
The activities gave me clarity about this.
aren’t relevant now, but may be in some future context
I’ve accumulated notes and understandings for that future context.
I know I’ll see the blip again if I mention a related topic.
can’t be addressed in one sitting
Though this example could be addressed in one sitting, I probably wouldn’t want to—it’s not important enough.
So the system smeared the task out over time.
Compared to naive spaced repetition
I’ve tried to solve this kind of problem before. For example, I’ve created a spaced repetition-style writing inbox, so that I can capture a blip like “what principles am I ‘inventing on’?” and repeatedly return to it over time. It’s a nice idea. But in practice, I found that when a prompt like this came up, I’d rarely want to engage.
One problem was the sameness of the prompt. I’d think “ugh, this again.” Faced with the same blank page I saw last time, I often don’t have anything new to say.
Another problem was decontextualization. I’d capture some idea or question while I was thinking about a particular problem. But they’d often resurface when I wasn’t thinking about that problem anymore. I’d feel emotionally disconnected, and wouldn’t feel like engaging.
A related final problem was the “endgame”: I’d develop my idea a bit, and then I’d reach the point where I was done with it for the moment. I didn’t want to see it regularly, but it didn’t feel safe to just archive my note, because then I felt I’d never see it again.
Two key ideas in our new system help address these problems.
The first is the system’s scaffolded microtasks. When a blip resurfaces, it’s not just showing you the same old prompt again. The system offers some suggested next actions, and possibly some new information to react to. As much as possible, we make those actions have time limits (“set a timer for two minutes and…”), a clear sense of the progress you’ll be making (“let’s make a list of examples…”), and clear goals (“add at least three examples…”). We’re accumulating a library of microtask patterns, and connecting them into playbooks that make sense for different types of blips.
The second key idea we call contextual resurfacing. When a blip shows up in your feed, it includes links to other related blips—possibly old blips which wouldn’t otherwise appear for a long time. Sometimes those related blips will actually be part of a suggested microtask (“how are these related?”). You can also explicitly specify topics which should trigger a snoozed blip to resurface. These elements aim to help the decontextualization and endgame problems by making it easy to double down on a juicy context, or to snooze a blip until you’re thinking about something in that context again. The net effect is a feeling of safety: you see each blip resurface many times, and you know you’ll see it again.
Both of these ideas depend on recent advances in large language models (LLMs). Still, we take a different approach to LLMs from most “productivity”-adjacent systems. Such systems tend to focus on using LLMs to do the task. But LLMs can’t really “do” any of the example blips I mentioned. As Taylor has charmingly put it: where other systems are about you prompting the LLM, this system is about the LLM prompting you.
What’s next
Taylor and I have been prototyping this system in classic Wizard-of-Oz style: we send our blips to a special email address. Then we each “play AI” for the other person, sending replies to resurface blips with microtasks. The recipient can “do” the microtask by replying, and so on. This has been remarkably generative! We can identify and reuse patterns which we imagine building into an LLM pipeline, but we can break out of those patterns whenever we want to experiment with something new.
I mention our approach here because I meet many technologists who immediately jump to building LLM infrastructure, then get stuck. Unless your domain needs realtime interaction, there’s no point in automating something until you know it actually solves the problem. It’s much easier to iterate on the conceptual design with manual human interaction. Once you have something which works so well that you want more of it than you can provide manually, that’s a good time to try introducing automation.
We’ve now reached that point with our prototype system. In many cases, we’re quite excited about how the system’s design has helped us push our blips forward. But as the “database” grows, and as we want to throw more material into it, we’re having trouble keeping up with the increasingly intricate scheduling and contextual resurfacing demands. We’ll soon need to introduce elements of automation.
We also have a huge number of open conceptual design problems to address, including many relating to system details not described in this letter. It’s all a bit overwhelming, but I’m thrilled that we’ve finally broken through to some promising uncharted territory for programmable attention.
------
My thanks to Taylor for his generous spirit of collaboration on this project, and for helpful suggestions on this text.
Comments
Benjamin Wheeler
This is giving me all sorts of ideas! Currently, I use a hodgepodge of Todoist todos (real example: "Look at Eric email and think about how he's using Replit; go to replit and try to make a relay of some sort"), Evernote notes (I'll have a creative project distributed across many notes, and in a summary note I'll list open questions to think about) and paper notebooks (which I rarely revisit once I've gotten a few pages further).
Bart Hoebink
Wow this is amazing! I think this shows a great use case of LLMs enhancing our thinking, instead of automating it (defeating its main purpose). If you'd like the to experiment with automation & LLMs, but want it low-code, completely private, scalable, and free, I couldn't recommend anything but N8N.io It keeps me lean and allows me to iterate very quickly Looking forward to seeing the idea being flushed out. And if you're not gonna build it, I will because this idea deserves an application. Thanks for the great content as always!!!
