Samuel
@samuel.fm
ok, let's break it down. at the core of atproto is the data. it lives own your own computer (or someone hosts it for you). think of it as a hard drive with JSON files on it data is structured (i.e. has a type), and can references other records on other peoples' computer with a URI like a hyperlink
Diagram labelled network state Two areas, my data storage and your data storage In my data storage, there is a post and a like. the like has an arrow to a post in your data storage
Lu wilson@todepond.com· 2 Nov 2024
okay let me get this straight... it's like this, right? to participate in this particular microblogging site we're all on, it has to go through the first-party appview?
75 184 654
Samuel
ok, let's say we want to see some posts. we can ask our PDS for some records, but that's not very interesting because it's just my stuff. we have a hyperlink to the post I liked, but that's a bit inconvenient because I'd need to go all the way to Lu's machine to see what it is
The above diagram with an arrow pointing to my data storage written beneath the arrow is "curl https://pds.mozzius.dev/fictional.getAllRecords" and beneath that is content from my data service again 2 1 76
Samuel
in order to make sense of the network, we're going to build a cache of all the content, so we can query it without needing to check each machine individual. think of it like Google's search index, except instead of googlebot scraping all the data, the PDSes offer a websocket that sends data instead
The data storage boxes now have "websocket" arrows pointing to a database symbol labelled "network cache" 3 1 81
Samuel
ok, we're getting somewhere. but to make sense of this big pile of data, we're going to offer some opinionated APIs. for example, a "getTimeline" API that returns all "post" records, with a computed count of how many "like" records are pointed at them now, we can fetch a timeline!
Now the "network cache" is in a box labelled AppView. also inside the appview is a box labelled "machine that calculates vote counts" beneath that is curl for "https://api.bsky.app/getTimeline" which returns posts from both data storages, and one of the post has a field of "likes: 1" 2 7 106
Samuel
this "AppView" service provides a "view" of the network. It will ignore records it doesn't know or care about, and derive useful information from the data it has now, since the data just sits on the web, and AppViews are downstream of the data, you can run another one and arrive at the same view!
The previous diagram, except the AppView section has been duplicated with the duplicate labelled "Alt AppView" 2 7 89
Samuel
so, imo the network (with multiple AppViews in play) looks like this. when you use a client, you ask an AppView to give you an opinionated view of the whole network. It acts as a cache in front of any number of hosts, which unlocks stuff like global search and accurate like counts (wild, right!)
Same diagram as the last one except the curl command has been replaced with a box with a butterfly emoji in it labelled "App" 6 4 80
Samuel
and to wrap things up, this is what writes look like. your client writes the new record to your PDS, which flows out down to the AppViews' caches, where it'll finally reach your client again (we're thinking of letting AppViews make writes on your behalf, but that's an optional UX/DX optimisation)
A green arrow labelled "write" goes from the App to your Data Storage at the top of the diagram. the Websocket arrows from your PDS to the AppViews are colored green along with the databases in the AppViews 7 16 97
Samuel
the "relay" is probably a red herring when thinking about the shape of the network. its just a many-to-one websocket service with some caching, so consumers that want to watch all events on the network (to make a feed, or whatever) can just watch the one websocket instead of 700+ as of Nov 2024
2 1 63
Samuel
running a PDS (your data store) is cheap. $5 VPS cheap, can have loads of users on the same host, and is immune to how busy the network is. if we gain 100mil users, you'll still be able to use a $5 VPS because it's only your data and only your traffic
4 7 104
Samuel
running a relay is more expensive but affordable (see whtwnd.com/bnewbold.net... ) running an AppView is tricker. this is a high-scale high-throughput service that requires seriously beefy machines. we don't expect people to be running them for fun on an Rasp. Pi, but we do want it to be feasible
2 4 73
5 more replies
Maël
Is there a list of appviews ? What would be the incentive for a organisation to create an appview, appart from decentralisation ?
3
Zetaphor
I realized I didn't leave a link, sorry! lowendbox.com/blog/1-vps-1... I am running two boxes from Racknerd, one of which is my PDS. The PDS box had its RAM upgraded bringing it from $12/year to $23/year
$1 VPS – 1 USD VPS Per Month (Updated October 2024) - LowEndBox
Can you really get a VPS for only a dollar a month...and if you can, how reliable are the hosts? Believe it or not - and you don't have to believe us because we've got links for you! - there are a nu...
lowendbox.com
 1 1 2
1 more reply
Olivier Forget
This is such a key difference compared to AP. AP just doesn’t scale well. I suspect that’s why you’ll eventually get users with huge followings here but not so much there.
3
andwhyisit
I want to self-host my pds one day, but I wanna use Rocky Linux as the distro so that I can use my existing vps. Is there a red hat installer yet?
C
Do you think PDSes might end up with an allowlist of friendly relays? For example, if a state actor DDoSes PDSes by asking for lots of backfills and/or subscriptions
1
Ahmed Soliman
so is the actual atproto that we reference just the protocol for data hosting that the PDSes run on? the AppViews sound like they could potentially be implemented differently based on a different client's requirements
4
Blisterexe
so, can i interact with bsky users without using any of bluesky's infra? I already use my own pds, but could i also use my own relay and appview, and still be able to, say, post this comment?
2
Bohdan Pavuk
UDF is broken, I critically disapprove of that. AppViews should get the ability to write to PDS's ASAP. you wouldn't believe the consequences UDF violation has in native Android development if I told you
mehiel
The UX/DX is about updating the current AppView's cache instantly for example and avoid the network delay at the place that the user interacts with at the time?
1
mehiel
It will stay as the source of truth. The update will reach the PDS finally as it is now but it will go through the AppView which enables it to instantly update it's data as well and reflect the changes in the UI.
1
1 more reply
jack
btw how do you get the Bluesky app to listen to another AppView? or would you need to fork the app and change that manually or smth
1 2
Lance Haig
Could you have an AppView that is only managing data for a subset of users? E.g . Part of a specific community?
1
Lance Haig
Thinking about his a bit more: If you had a website that used pds data from users and you wanted to make it easy for others to see this data and for the owner of the data to control who sees that data. Would you need an AppView?
1
Tom Sherman
AppView is needed to perform and cache aggregations. Imagine fetching and paginating over every record in every PDS within your bsky mini network to get the like count. This would be super slow, inefficient, and expensive for everyone involved
1
Pablo GPS
Is it not possible to restrict who access the data? For example to stop competition. And if all data is public this means blocks can't really be enforced, right?
Jason Judge
So the websockets are like a permanently open pipe that data can be squirted down at any time and picked up at the other end, without the overhead of opening and closing http connections?
Michael
📌 okay you’ve finally driven me to learn how to use these pins for this
1
Michael
For anyone else wanting to acquire such magical power, pin the feed below to your homepage. It will build up a list of all the posts you have replied to with a pin emoji. bsky.app/profile/did:...
Deleted post
1
More replies on Bluesky