Samuel
@samuel.fm
btw, I would love to read a tutorial on how to make a proper algorithmic feed. I wouldn't really know where to start
2 1 22
Samuel
in that direction - certainly a level above regex-y ones. Like I don't really comprehend how you're supposed to figure out what's relevant, like I could imagine how you'd do it for one specific person but figuring it out on the fly for everyone who might possibly look at your feed? no clue
2 8
David Buchanan
it's out of my wheelhouse really but iiuc you first want a really big database, then start doing "collaborative filtering"
1 3
David Buchanan
you probably want to start calculating correlation coefficients between users based on their like/follow histories (you can treat them as really big vectors)
2 3
David Buchanan
I think the hard part is deciding which stuff to precompute (likely in large batches), and which to do on-the-fly during the materialisation of a particular feed
2
Brian Gawalt
me, i would: a) restrict myself to one language at a time b) keep one global model for all users in that language, and keep one going for each subscriber c) build a huge table of {user, post, bools for liked and reposted} d) define Happy Points scale: 1 for a like, 10 for repost
1 2
Brian Gawalt
e) generate the global model aggregating over all users, and predict from post features to Happy Points. I would start with a random forest due to easiness: scikit-learn.org/stable/modul... f) do the same for the user-local models, except only look at that user's rows in the Big Table
sklearn.ensemble.RandomForestRegressor
Examples using sklearn.ensemble.RandomForestRegressor: Release Highlights for scikit-learn 1.4 Release Highlights for scikit-learn 0.24 Combine predictors using stacking Comparing Random Forests an...
scikit-learn.org
 1 2
Brian Gawalt
g) predict Happy Points as a weighted average of Global and Local predictions, with weight accruing to Local as you have more rows in the Big Table from that user. h) Each refresh, rank all recent never-before-presented posts by predicted Happy Points. Log what youve shown to build up the big table!
1 2
Brian Gawalt
i) finally, check which posts got likes/reposts, get that feedback into the Big Table from there -- that's it. tweak the knobs around (how recent is recent enough to rank? how should Global-Local weight shift with user row count?) in randomized A/B tests of different settings
1 2