Remix.run Logo
ivraatiems 9 hours ago

Kind of Google to create a market opening for its competitors like this. I hope Kagi, Bing, and DuckDuckGo are taking notes.

data-ottawa 7 hours ago | parent | next [-]

reCaptcha is a pretty strong wall to allow only Google to index websites, especially now that you need device verification. Throw in Cloudflare too.

There’s not much room to squeeze in when your competitors hold the keys to 15 million top websites.

xmcp123 5 hours ago | parent [-]

I write a lot of scrapers. Both of those are pretty trivial to bypass at scale.

HDBaseT 2 hours ago | parent [-]

What about not at scale?

I find it wild that "at scale" we can bypass anti-bot measures, but just "normal" internet use (i.e Non-Google Browser or VPN) will throw a million captchas at you.

cgnat is pretty bad too.

nostrademons 9 hours ago | parent | prev | next [-]

The problem is that the web as we know it (useful, human-curated information that's put out there to help people) is also over. It's been totally overrun with AI slop. Even before AI could be used to create propaganda on a scale that we could only dream about 5 years ago, it's been declining under the weight of SEO sweatshops for a good 10 years. Meanwhile the actually decent content, the individual hobbyists who are just sharing their knowledge, have largely left under the weight of comment spam and DDoS attacks and doxxing.

So if another search engine does arise, it won't find anything useful, because the useful content on the web has been buried under slop, and largely removed. Your best bet today is a curated directory, sorta like the original Yahoo, where you allowlist the web to only real sites, download them, and make them searchable. I think this is actually Kagi's approach. But the open web as we knew and loved it is dead.

ares623 6 hours ago | parent [-]

Curators will become desirable again. The Devil Wears Prada 3.

torben-friis 9 hours ago | parent | prev | next [-]

My literal first thought was "do I seriously need to use bing now?".

Supermancho 9 hours ago | parent | next [-]

Bing has been better than Google for some time. Again, it's embarrassing for them to sacrifice marketshare for paid results and an intermediate-form AI fad that will turn into the same paid result funnel.

mrweasel 9 hours ago | parent | next [-]

Bing is surprisingly not to bad. I don't use it anymore, but it's been providing better results than Google for sometime.

RyanOD 9 hours ago | parent | prev [-]

I hear people cite other search engines as "better" all the time. Better how?

edelbitter 8 hours ago | parent [-]

e.g. for a two keyword search, Google & DDG return results containing a similar (but more at the moment, more popular, so I understand why they do this) keyword as the first one, and no relation whatsoever with the second. Any search that manages to actually show results related to both of my input terms get the "better" award from me.

sphars 9 hours ago | parent | prev | next [-]

There is a 99% chance (IMO) that Microsoft is going to go the same route as Google here

vitorsr 5 hours ago | parent [-]

Microsoft has already gone down this road some three years ago...

https://blogs.microsoft.com/blog/2023/02/07/reinventing-sear...

tedd4u 8 hours ago | parent | prev | next [-]

DuckDuckGo uses the bing index/backend. I’ve had it as default for 5-8 years. Probably once a day I’ll add the !g to pop it over to Google. Works great. I search a lot, many different types of queries. When I pop over to Google it’s usually a Boolean query looking for a needle in a haystack (that one comment somewhere where someone is using the same combination of two or three rare items together).

BrunoBernardino 8 hours ago | parent | prev [-]

While there are good options like DuckDuckGo, Mojeek, Ecosia, there are plenty of (better) alternatives, where you're not the product [1], I'd recommend looking into!

[1]: https://alternativeto.net/software/google-search/?license=co...

Zigurd 9 hours ago | parent | prev | next [-]

I'm sure there's a niche for a product for search nerds. Something that leans into inverted indexes like the classic Lexis/Nexis search. But it's got to have Google-like coverage.

cortesoft 9 hours ago | parent [-]

Niche + Google-like coverage is not very economically viable. To store and update a search index of that size requires a lot of resources, and being niche means you don’t have a lot of resources.

Very few of the smaller search engines actually do their own indexing for exactly this reason.

edelbitter 8 hours ago | parent [-]

I wonder if the same coverage as before is now more economically feasible. The internet has gotten .. smaller, lately.

raincole 8 hours ago | parent | prev | next [-]

Kagi relies on Google search.

baggachipz 8 hours ago | parent | next [-]

True in large part, but they've been diversifying their providers in the expectation that Google shut everybody out.

hootz 7 hours ago | parent | prev | next [-]

But the results are still 1000x better than Google's. Something is being done there.

dgellow 7 hours ago | parent | prev | next [-]

Sure but we are talking about the UI here, not the index being used

raincole 7 hours ago | parent [-]

But if Kagi manages to become a serious competitor in the search engine space, Google will cut them off from their index. Why will not they?

dgellow 6 hours ago | parent [-]

It’s possible but they would need to be so massive to even just start making a dent in google market share. And Google hasn’t blocked larger search engines from using their index

AndroTux 7 hours ago | parent | prev [-]

They mostly use Bing, at least from my testing.

xerox13ster 9 hours ago | parent | prev | next [-]

I've been using Startpage as my default search engine for a while now for any search where I actually need information and not sales or marketing bullshit.

When I use google, usually from my phone, I am reminded of why I don't use google on desktop.

With the announcement of this move by them, I just manually removed google as an address bar search engine option in all my browsers on desktop and mobile.

kylehotchkiss 7 hours ago | parent | prev [-]

Cloudflare seems like they have the capability to take this on.

Human produced content should be separated from sites primarily hosting slop. That seems solvable?