Samuel
@samuel.fm
in that direction - certainly a level above regex-y ones. Like I don't really comprehend how you're supposed to figure out what's relevant, like I could imagine how you'd do it for one specific person but figuring it out on the fly for everyone who might possibly look at your feed? no clue
2 8David Buchananit's out of my wheelhouse really but iiuc you first want a really big database, then start doing "collaborative filtering"
1 3David Buchananyou probably want to start calculating correlation coefficients between users based on their like/follow histories (you can treat them as really big vectors)
2 3David Buchananand then you can do "users similar to you liked these posts"
2 3David BuchananI think the hard part is deciding which stuff to precompute (likely in large batches), and which to do on-the-fly during the materialisation of a particular feed
2David Buchanancould maybe cluster users in some n-dimensional space, like jaz's atlas, then look at their nearby users
1
Brian Gawaltme, i would:
a) restrict myself to one language at a time
b) keep one global model for all users in that language, and keep one going for each subscriber
c) build a huge table of {user, post, bools for liked and reposted}
d) define Happy Points scale: 1 for a like, 10 for repost
1 2Brian Gawalte) generate the global model aggregating over all users, and predict from post features to Happy Points. I would start with a random forest due to easiness: scikit-learn.org/stable/modul...
f) do the same for the user-local models, except only look at that user's rows in the Big Table
sklearn.ensemble.RandomForestRegressor
Examples using sklearn.ensemble.RandomForestRegressor: Release Highlights for scikit-learn 1.4 Release Highlights for scikit-learn 0.24 Combine predictors using stacking Comparing Random Forests an...
scikit-learn.org
Brian Gawaltg) predict Happy Points as a weighted average of Global and Local predictions, with weight accruing to Local as you have more rows in the Big Table from that user.
h) Each refresh, rank all recent never-before-presented posts by predicted Happy Points. Log what youve shown to build up the big table!
1 2Brian Gawalti) finally, check which posts got likes/reposts, get that feedback into the Big Table
from there -- that's it. tweak the knobs around (how recent is recent enough to rank? how should Global-Local weight shift with user row count?) in randomized A/B tests of different settings
1 2Brian Gawaltive totally omitted how you turn "details about the user" and "details about the post" into the array of float that the regression framework is going to want. let me plug my company and say here's how you can convert natural language to such an array: cloud.google.com/vertex-ai/do...
Get text embeddings | Vertex AI | Google Cloud
cloud.google.com