Since most Lemmy instances won't come loaded with Google analytics, solely relying on Google crawlers I would imagine will be very ineffienct at gathering all the posts and comments.
You don't need Google Analytics to be crawled. You need Google Search Console integration to improve crawling.
So is there a way to search across instances?
Seriously, this will distance google from usefulness even further. Chatbots already at the jugular of google right now.
Can the big AI companies crawl and harvest lemmy and fediverse?
To comment on your last question. In theory, yes it can, but practically it will be as tricky as Google's method. It will be much more user-friendly for end-user though, you just have to ask "Is the housing market about to burst? fediverse top posts" and I would imagine the results will be pretty relevant.
