Samuel
dogfooding the thread composer tonight
Samuel@samuel.fm· 3 Nov 2024
ok, let's break it down. at the core of atproto is the data. it lives own your own computer (or someone hosts it for you). think of it as a hard drive with JSON files on it data is structured (i.e. has a type), and can references other records on other peoples' computer with a URI like a hyperlink
Diagram labelled network state Two areas, my data storage and your data storage In my data storage, there is a post and a like. the like has an arrow to a post in your data storage
2 1 37
Samuel
ok, let's break it down. at the core of atproto is the data. it lives own your own computer (or someone hosts it for you). think of it as a hard drive with JSON files on it data is structured (i.e. has a type), and can references other records on other peoples' computer with a URI like a hyperlink
Diagram labelled network state Two areas, my data storage and your data storage In my data storage, there is a post and a like. the like has an arrow to a post in your data storage
Lu wilson@todepond.com· 2 Nov 2024
okay let me get this straight... it's like this, right? to participate in this particular microblogging site we're all on, it has to go through the first-party appview?
75 184 654
Samuel
ok, let's say we want to see some posts. we can ask our PDS for some records, but that's not very interesting because it's just my stuff. we have a hyperlink to the post I liked, but that's a bit inconvenient because I'd need to go all the way to Lu's machine to see what it is
The above diagram with an arrow pointing to my data storage written beneath the arrow is "curl https://pds.mozzius.dev/fictional.getAllRecords" and beneath that is content from my data service again 2 1 76
Samuel
in order to make sense of the network, we're going to build a cache of all the content, so we can query it without needing to check each machine individual. think of it like Google's search index, except instead of googlebot scraping all the data, the PDSes offer a websocket that sends data instead
The data storage boxes now have "websocket" arrows pointing to a database symbol labelled "network cache" 3 1 81
Samuel
ok, we're getting somewhere. but to make sense of this big pile of data, we're going to offer some opinionated APIs. for example, a "getTimeline" API that returns all "post" records, with a computed count of how many "like" records are pointed at them now, we can fetch a timeline!
Now the "network cache" is in a box labelled AppView. also inside the appview is a box labelled "machine that calculates vote counts" beneath that is curl for "https://api.bsky.app/getTimeline" which returns posts from both data storages, and one of the post has a field of "likes: 1" 2 7 106
Samuel
this "AppView" service provides a "view" of the network. It will ignore records it doesn't know or care about, and derive useful information from the data it has now, since the data just sits on the web, and AppViews are downstream of the data, you can run another one and arrive at the same view!
The previous diagram, except the AppView section has been duplicated with the duplicate labelled "Alt AppView" 2 7 89
Samuel
so, imo the network (with multiple AppViews in play) looks like this. when you use a client, you ask an AppView to give you an opinionated view of the whole network. It acts as a cache in front of any number of hosts, which unlocks stuff like global search and accurate like counts (wild, right!)
Same diagram as the last one except the curl command has been replaced with a box with a butterfly emoji in it labelled "App" 6 4 80
Samuel
and to wrap things up, this is what writes look like. your client writes the new record to your PDS, which flows out down to the AppViews' caches, where it'll finally reach your client again (we're thinking of letting AppViews make writes on your behalf, but that's an optional UX/DX optimisation)
A green arrow labelled "write" goes from the App to your Data Storage at the top of the diagram. the Websocket arrows from your PDS to the AppViews are colored green along with the databases in the AppViews 7 16 97
Samuel
the "relay" is probably a red herring when thinking about the shape of the network. its just a many-to-one websocket service with some caching, so consumers that want to watch all events on the network (to make a feed, or whatever) can just watch the one websocket instead of 700+ as of Nov 2024
2 1 63
Samuel
running a PDS (your data store) is cheap. $5 VPS cheap, can have loads of users on the same host, and is immune to how busy the network is. if we gain 100mil users, you'll still be able to use a $5 VPS because it's only your data and only your traffic
4 7 104
Samuel
running a relay is more expensive but affordable (see whtwnd.com/bnewbold.net... ) running an AppView is tricker. this is a high-scale high-throughput service that requires seriously beefy machines. we don't expect people to be running them for fun on an Rasp. Pi, but we do want it to be feasible
2 4 73
Samuel
any future apps being able to “do a threads” I think is a really compelling reason to build on atproto
Samuel@samuel.fm· 2 Nov 2024
you can create a new service using the same technology, and it will have the same benefits. what’s interesting is that since the data is structured and open, you can “borrow” data from other services - e.g. you can kickstart your algorithm with the existing social graph
1 38
Samuel
ok how about this: AP: you have multiple twitters that talk to each other AT: what if twitter drew its data from the open web?
dan@danabra.mov· 30 Oct 2024
i wonder if diagrams like these might help draw the distinction between ActivityPub (and how Mastodon uses it) and AT Protocol (and how it’s used by Bluesky and atproto apps)? with AP, instance = application + hosting. data gets sent between instances. with AT, data flows *from* hosting *to* apps
diagram with multiple application+hosting boxes. each box contains some users and some posts. data is sent between the boxes.a diagram with two layers. in the top layer, users choose hosting (and their data is stored in their hosting). the data then flows down from hosting into separate apps which mirror and aggregate the data they receive from hosting. the hosting boxes are drawn with dashed borders to indicate they're not fixed, and the user may move their data between different hosting without affecting the applications.
14 56 291
Samuel
AP’s credible exit: you can make a new twitter. however your data is stuck on the old one AT’s credible exit: the data layer is independent from the apps and the network state is reproducible, so you can move to a new twitter seemlessly and even with an adversarial data host
2 9 79
Samuel
bluesky is a “view” of all bluesky-type data on the open web. making a new “instance” of bluesky is to create a new “view” of the same data. this is like how Google and Bing have created independent search indexes of the same data
3 16 96
Samuel
atproto data is a lot more structured, so two different “view”s of all bluesky-type data will be nearly identical. this is why it’s called an “AppView”
2 4 58
Samuel
you can create a new service using the same technology, and it will have the same benefits. what’s interesting is that since the data is structured and open, you can “borrow” data from other services - e.g. you can kickstart your algorithm with the existing social graph
1 3 51
Samuel
free likes on test posts, except what I'm testing is a bug which sends many many posts at once by accident = a lot of likes
6 2 113
Samuel
it’s feeling particularly cozy here the past couple days. liking it :)
12 264
Older posts