Samuel
@samuel.fm
surprisingly controversial, just want to say two things 1. 100% depends on execution. If the model is bad then obviously it won’t be helpful 2. It is obviously no substitute for good human-written alt text. HOWEVER, even though this site has a good culture for writing alt text, only a tiny…
Samuel@samuel.fm· 1 Jun 2024
huge!
Mozilla Developer @mozhacks Firefox 130 is bringing a game-changing feature: automatic alt-text generation for images using a fully private on-device Al model! Initially available in the built-in PDF editor, our aim is to extend this to general browsing for screen reader users.
7 4 68
Samuel
…fraction of images posted even has alt text, and I’d bet that only a fraction of that have *good* alt text. While this tool wouldn’t raise the ceiling, it certainly would provide a baseline for that overwhelming majority of images that would otherwise go without. Please don’t stop writing alt text!
1 1 37
Samuel
and like, sure, probably going to be a lot of slop, but is this really *worse* than nothing? bare in mind this is on-device, aka your browser looking at an image without alt text and describing it, rather than the person uploading it adding lazy alt text
Conversation with chatgpt where I asked them to provide alt text for the famous “Washington crossing the Delaware River” painting appropriate for a microblogging site ChatGPT: George Washington and his troops crossing the icy Delaware River during the Revolutionary War, depicted in a historic painting. 2 1 26
Samuel
I am definitely anti-automatic alt text generation at upload time without human review. that seems bad, but also that’s not what Firefox is proposing
2 32
neko
i would really love an upload version but with mandatory human review by the uploader... like, pre-fill alt text for me but have me change and approve it. would make things so much simpler.
3
fellowship of the prim
Agreed. And while this isn't an excuse for most people, this is still a better description than some people with intellectual disabilities could write themselves. If executed well, it could be a good accessibility tool in both directions--which is AI at its best.
2
Bossett
😬 I don’t think you’re wrong, but the creeping adddition of AI into everything, the provenance of training sets, etc. is more where the problem lies. It also isn’t really alt that *describes*, it’s alt that *describes and contextualises*. Kind of undermines a lot of ‘how to do this right’ effort.
3 1 15
Bossett
I think these are examples of good applications of the tech - I wish these were the early examples and not chatbots - but also not convinced (and this may be a ‘yet’) of their ability to actually do this in a way that would describe most content users use to interact with each other.
1 6
Samuel
yeah, I don’t disagree… I just find the overall stats of % images with alt text depressing :/
1 1 8
Bossett
start advertising that images without alt will be fed to the machine 👀
Bossett@bossett.social· 28 Mar 2024
Pretty soon (well, right now, but niche) leaving alt-text off your images is going to directly result in them being fed into an AI describer, as the only way for some people to experience your content.
2 6
Samuel
amusing, but images with alt text might get fed into the Common Crawl datasets. you just can’t win 🤷
4 5
Samuel
at some point we might need to start faxing our memes to the group chat with a hand-written note describing it
2 8
Samuel
I broadcast my memes over a soviet Numbers station left over from the cold war
4
Biscuit
changing your label from “Sam” to “Sam the Faxer” in my in-progrsss labeler
2
Scaredyfish 🐟🐡🐠
Well, if people write quality alt text, and that is used as training data to improve the AI stuff, that might be good. On the other hand, there would need to be a way to distinguish the automatic stuff so it’s not training on its own outputs.
3
Metaflame
What I use as "advertising" for using alt text is improved searchability & the 1000-char limit for extra context & description details, both of which are incredibly beneficial to artists. I feel like those are great selling points for people who want their content to be seen & engaged with more.
1
goeo like the
alt text definitely does not need to contextualise, imo. yes i often do it, but if you can see the image you should _not_ be getting more out of reading the alt text
1 4
Bossett
the contextualisation I'm talking about is things like 'this is Captain America in a scene from the Avengers' instead of just 'this is a man dressed in a blue superhero outfit'
2 5
goeo like the
well someone that hasn’t seen avengers wouldn’t necessarily know that from just the screenshot would they?
1 2
Bossett
yeah - but imagine I post this with some other text, if you have no visual reference you need the context that this is a meme format to make sense of it Like you can describe it exactly, but without lending it meaning that the author intends
Captain America from the Avengers movie saying "I understood that reference" 1 2
goeo like the
i think it’s okay to not understand stuff sometimes, alt text is meant to have it be equitable for people with vision impairment, not necessarily make it make sense, imo
1 4
The Doctor (Commissions open)
I feel, if we have the tool, which is not cluttering the usual UI, additional context in alt is great. I see so many posts (movie screengrabs) where a lot of users mention the movie it's taken from, or sometimes even more context what was happening... I try to do the same as well.
2
Tom
If image description was implemented automatically, even if it was marginally acceptable, it would still be a big boon for the visually impaired. If Bsky could auto-generate alt-text, then ask a user to review it and add context if needed, it would be great!
1 2
Bossett
yeah 100% - maybe wasn't super clear, I'm saying it remains to be seen how acceptable it is because the examples you can find are describing scenes 'as is' and not 'oh this meme is a scene from', etc. (or whatever) which it may be fine at, but dunno.
1
goeo like the
i think this is not just good, but necessary. editing ai output to fix it will likely be a lot easier than writing it yourself from scratch, and for everyone not willing to spend time writing alt text it provides a base line. we _exclusively_ use ai for translation, and that’s ok, for example
2 6
Samuel
you’re forgetting translation models were made before ML got rebranded as AI, and are thus exempt from AI discourse
1 9
goeo like the
i’ve said this before, but having enabled the require alt text while posting images option definitely makes me avoid posting images if i can, it’s just like, work, and i don’t think any of my followers have accessibility needs anyway (so i’m doing it for strangers, even when replying to mutuals)
4
str4d
Exactly this. And the app can detect that a user made no edits, and mark the text as auto-generated so users who don't want that can have their app ignore it, treating the image as having no alt text (the current base case).
str4d@str4d.xyz· 3 May 2023
- Image description tools. We have the tooling to enable the equivalent of a translation app. And if Bluesky clients could scan an image and propose alt text, it would make some people more likely to provide alt text, as they'd be editing the suggestion instead of writing from scratch.
3
Time Spinner
if only there was some site with millions human-written and editor-vetted descriptions for real world images about hundreds of thousands of topics available for free in hundreds of languages. But where would you find such a site?
1 2
Seithon
See.. this is an example of AI being useful and helpful... I just hope it's not using stolen stuff :/