Jaz
We moved actor search to Open search to reduce load on the AppView database but it's definitely got some quirks. It did successfully give us back headroom on those DB replicas and smooth out CPU spikiness but yeah. Probably gonna be a new search backend for actors with follow context next year.
1 1 17
Jaz
Typeahead is supposedly using 2 and 3 ngram indices based on the display name and handle but it seems to really prefer matching handles only. Without typeahead it searches in the entire profile record which also doesn't feel great, it should prio Handle -> DN -> Bio and have different fuzziness
1 8
Jaz
Current version of search also doesn't know about anything like how many followers someone has or how many posts they've made and has no way to prioritize more active accounts. Lots of room to improve in a custom search service but it requires time/effort we don't have capacity for atm to scale.
2 9
Jaz
Any solution we'd build now we'd want to scale to 100M users which would require a good amount of planning/testing to make sure things like celebrities etc. don't just completely break the scaling of the service. We can do it but it takes time that needs to go into v2 arch right now.
1 1 7
Jaz
The magic numbers we always use is, imagine a user has millions of posts, has given millions of likes, follows hundreds of thousands of accounts, and is followed by 100M accounts. That's not gonna be every user but that's the extreme of what we need to support.
1 9
Samuel
@samuel.fm
👍 thanks for the in depth explanation! From what I’ve seen, Paul’s clientside hack seems like it’ll be able to tide things over until you’re able to get to it, whenever that happens
5