ok, let's break it down.
at the core of atproto is the data. it lives own your own computer (or someone hosts it for you). think of it as a hard drive with JSON files on it
data is structured (i.e. has a type), and can references other records on other peoples' computer with a URI like a hyperlink
okay let me get this straight... it's like this, right?
to participate in this particular microblogging site we're all on, it has to go through the first-party appview?
ok, let's say we want to see some posts. we can ask our PDS for some records, but that's not very interesting because it's just my stuff.
we have a hyperlink to the post I liked, but that's a bit inconvenient because I'd need to go all the way to Lu's machine to see what it is
in order to make sense of the network, we're going to build a cache of all the content, so we can query it without needing to check each machine individual.
think of it like Google's search index, except instead of googlebot scraping all the data, the PDSes offer a websocket that sends data instead
ok, we're getting somewhere.
but to make sense of this big pile of data, we're going to offer some opinionated APIs. for example, a "getTimeline" API that returns all "post" records, with a computed count of how many "like" records are pointed at them
now, we can fetch a timeline!
this "AppView" service provides a "view" of the network. It will ignore records it doesn't know or care about, and derive useful information from the data it has
now, since the data just sits on the web, and AppViews are downstream of the data, you can run another one and arrive at the same view!
so, imo the network (with multiple AppViews in play) looks like this. when you use a client, you ask an AppView to give you an opinionated view of the whole network. It acts as a cache in front of any number of hosts, which unlocks stuff like global search and accurate like counts (wild, right!)
and to wrap things up, this is what writes look like. your client writes the new record to your PDS, which flows out down to the AppViews' caches, where it'll finally reach your client again
(we're thinking of letting AppViews make writes on your behalf, but that's an optional UX/DX optimisation)
the "relay" is probably a red herring when thinking about the shape of the network. its just a many-to-one websocket service with some caching, so consumers that want to watch all events on the network (to make a feed, or whatever) can just watch the one websocket instead of 700+ as of Nov 2024
running a PDS (your data store) is cheap. $5 VPS cheap, can have loads of users on the same host, and is immune to how busy the network is. if we gain 100mil users, you'll still be able to use a $5 VPS because it's only your data and only your traffic
running a relay is more expensive but affordable (see whtwnd.com/bnewbold.net... )
running an AppView is tricker. this is a high-scale high-throughput service that requires seriously beefy machines. we don't expect people to be running them for fun on an Rasp. Pi, but we do want it to be feasible
After reading through this, I definitely feel like I have a better appreciation for both what ATProto is and how the general network works behind the scenes with a client. The cache on a websocket "relay" is super nifty and seems like a really creative solution to optimize interactions
I've been extra curious how it works because of the whole "users are websites" deal and the ability to host your own PDS..my mind went straight to things like traffic and the costs for each user, BUT here it appears the PDS itself is relatively cheap to maintain vs the larger cache bsky interacts w/
yeah exactly. every user a website, but since data is cryptographically verifiable (Authenticated Transfer Protocol, see?), you don't need to fetch it directly from your machine to know it's really from you. this allows trustless network caches to sit between your machine and all the traffic
Lu wilson@todepond.com· 2 Nov 2024
