What are some of the niche search engines build on Google's index affected by this?

> “search the entire web”

TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.

It's been clear for the last decade that we have to wean ourselves off of centralized search indexes if only to innoculate the Net against censorship/politically motivated black holing.

I can only weep at this point, as the heroes that were the Silent and Greatest generations (in the U.S.), who fought hard to pass on as much institutional knowledge as possible through hardcore organization and distribution via public and University library, have had that legacy shit on by these ad obsessed cretins. The entirety of human published understanding; and we make it nigh impossible for all but the most determined to actually avail themselves of it.

I built my own web search index on bare metal, index now up to 34m docs: https://greppr.org/

People rely too much on other people's infra and services, which can be decommissioned anytime. The Google Graveyard is real.

This might be me reading it wrong, but isn't shutting down the full-web search going against the ruling mentioned in the Kagi post?

> Google must provide Web Search Index data (URLs, crawl metadata, spam scores) at marginal cost.

Maybe they're shutting down the good integration and then Kagi, Ecosia and others can buy index data in an inconvenient way going forward?

Excuse the self-promotion but Mojeek offers a web search API (>9 billion pages): https://www.mojeek.com/services/search/web-search-api/

It's a noble effort, but they're so late to the game that it's hard to see them making a significant dent. I hope I'm wrong.

They were:

> aiming to serve 30% of French search queries [by end of 2025]

https://blog.ecosia.org/launching-our-european-search-index/

Much easier said than done, especially if you are serving users on scale.

The beauty about Google Programmable Search across the entire web is that it's free and users can make money by linking it their Adsense account.

Bing charge per query for the average user. Ecosia and Qwant use Bing to power their results, probably under some type of license, which results in them paying much less per query than a normal user.

Bing recently shut down their API product, which was already very expensive.

If you want programmatic access to search results there aren't really many options left.

I can manage fine with other search indexes for English language searches; weather that is because others got better or google got worse i cannot tell, though I suspect the latter.

But for searching in more niche languages google is usually the only decent option and I have little hope that others will ever reach the scale where they could compete.

Bing's index is smaller than Google's, and anecdotally I get fewer relevant results when using it, particularly from sites like Reddit that have exclusive search deals with Google.

"Google will discontinue third-party niche search engine access to full-web search" would be far clearer.

Given that the title supplied is effectively editorialised, and the original article's title is effectively content-free ("Updates to our Web Search Products & Programmable Search Engine Capabilities"), my rewording would be at least as fair.

HN's policy is to try to use text from the article itself where the article title is clickbait, sensational, vague, etc., however. I suspect Google's blog authors are aware of this, and they've carefully avoided any readily-extracted clear statements, though I'll take a stab...

Here's the most direct 'graph from TFA:

Custom Search JSON API: Vertex AI Search is a favorable alternative for up to 50 domains. Alternatively, if your use case necessitates full web search, contact us to express your interest in and get more information about our full web search solution. Your transition to an alternative solution needs to be completed by January 1, 2027.

We can get a clearer, 80-character head that's somewhat faithful to that with:

"Google Search API alternative Vertex AI Search limited to 50 domains" (70 chars).

That's still pretty loosely adherent, though it (mostly) uses words from the original article. I'm suggesting it to mods via email at hn@ycominator.com, others may wish to suggest their own formulations.

Kagi doesn't have a partnership with Google - they work under adversarial interoperability, stealing results from Google against their will, and paying some third-party to enable this. They'd like to simply pay Google, but Google doesn't want their money.

Yeah that's where I started out in 2021. Been at it for almost 5 years now, last three of which full time. I'm indexing about 1.1 billion documents now off a single server.

Hard part is doing it at any sort of scale and producing useful results. It's easy to build something that indexes a few million documents. Pushing into billions is a bigger challenge, as you start needing a lot of increasingly intricate bespoke solutions.

Devlog here:

https://www.marginalia.nu/tags/search-engine/

And search engine itself:

https://marginalia-search.com/

(... though it operates a bit sub-optimally now as I'm using a ton of CPU cores to migrate the index to use postings lists compression, will take about 4-5 days I think).

Might find YaCy interesting. It’s meant to be a decentralised search engine where users scrape the internet and can search other users indexes in a kind of torrent like way.

I found it didn’t really work as a real search engine but it was interesting.

Good luck scraping websites without being blocked, if you're not Google.

This is the type of monopoly abuse these laws were designed to target, and antitrust laws actually do work against large companies.

If you actually enforce them.

Unfortunately, during the Reagan administration, political sentiment toward monopolies shifted and since then antitrust law has been a paper tiger at best.

Soon you'll find that you cannot exist on the web without relying on third parties. Sometimes you'll even have trouble getting paid thanks to the painful existence of payment processors.

That's why I eschew HTTPS.

Kind of, however the Google Search Bar present on website is usually there to search across their domain, the search results are limited to their domain e.g example.com/page1, example.com/page2. Google will carry on supporting this.

What they are ending is their support for websites to search across the entire web. The websites that search across the entire web are usually niche search engine websites.

There is literally thousands of independent search engines that use Programmable search to search the entire web. Many ISP providers use it on their homepage, kids-based search engines like wackysafe.com use it, also search engines that focus on privacy like gprivate.com etc

They'll go adversarial interop through SerpAPI, just like Kagi does. SerpAPI will get the money instead of Google getting it.

I suspect its going to hurt the indie developers and small start-ups who do not have special licensing agreements.