Remix.run Logo
SearXNG: A free internet metasearch engine(github.com)
111 points by theanonymousone 4 hours ago | 29 comments
asciimoo 2 hours ago | parent | next [-]

Ohi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister (https://hister.org/).

Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews and the full page content via MCP.

Take a look at how the MCP can be utilized: https://hister.org/posts/give-your-ai-assistant-a-private-me...

zeroq 14 minutes ago | parent | next [-]

I'm sorry for not taking the time to read the docs, but I have a question.

Some 20 years ago a friend of mine has set up a local proxy (python if I'm not mistaken) that was gathering all his web traffic and served him as a long term memory. The proxy had a web interface and allowed him to quickly find something he saw ca. 10 days ago, or that specific algorithm he recalls but can't remember it's name.

For years I've been collecting links to different work related trivia which I use on a daily basis as a rabbit-from-a-hat solution to answer random question from friends and coworkers. For example someone randomly asked me for an idea for color palette for data charts and I can immediately give them a scientific research into the color palette. Or an obscure algorithm.

But with time the collection has grown substantially and it's really cumbersome to find the proper things.

Would your project be a good fit for my problem?

justusthane 2 minutes ago | parent [-]

Also very interested in this. I was playing around with doing the same thing with YaCY. I want the proxy aspect so that I can proxy my phone traffic through it as well.

chrisss395 18 minutes ago | parent | prev | next [-]

I love your idea and wondered why saving and indexing browser visited pages was not being done. Does this handle large amounts of local files, for example 10-20TB across file types like Powerpoint, Excel, Word, and PDF?

asciimoo a minute ago | parent [-]

In its current form it cannot handle this amount of data (and doesn't support powerpoint/excel/word yet), but this is a valid use-case, I've added a TODO item to experiment with it.

kristianpaul an hour ago | parent | prev | next [-]

Is this similar to fastcrw ?

asciimoo an hour ago | parent [-]

Both are search engines, but that's all the similarity. Hister has a traditional crawler, but its biggest strength is automatically indexing browser tabs as those are rendered. This way it bypasses authentication, CloudFlare, captchas and most of the annoying limitations of traditional crawlers. Hister also provides full offline result previews. Check out the small read-only demo: https://demo.hister.org/

operatingthetan 2 hours ago | parent | prev [-]

I installed this a while back and honestly I almost never touch it. It turns out that for me searching my history doesn't really replace a search engine at all. The built in extractor list is pretty limited and adding them seems like too much of an ordeal for me to bother.

exiguus an hour ago | parent | prev | next [-]

SearXNG is my daily internet search now +5 years; with YaCY Backends and else as fallback. I also build internal document search or RAG applications with this setup (SearXNG also support json results). However, there are some downer I accept because of privacy: 1. Its slower and the results are not that good then with others. But fast and good enough for most of my queries. 2. From time to time you get blocked on the duckduckgo, brave or whatever search and you must solve some captures. You can prevent this by getting and using API-Keys from them.

The nice thing about using your own backend is, that you can prio it in the results and for example, if I crawl the smallweb and other site important for myself, this sites come up first in the results.

chatmasta 30 minutes ago | parent | prev | next [-]

I’ve always liked this tool, but I’m of two minds regarding the privacy gained by sending my searches to 280 companies instead of just one.

satvikpendem 3 hours ago | parent | prev | next [-]

TinySearch wraps this and works well for agents. It's better than the native SearXNG MCP because it optimizes the context before it even gets to the agent so as to not waste tokens.

https://github.com/MarcellM01/TinySearch

drnick1 3 hours ago | parent | next [-]

SearXNG did not include a built-in MCP server, last time I checked.

ProofHouse 3 hours ago | parent | prev [-]

Props

goodroot 2 hours ago | parent | prev | next [-]

This appears to be a key tool for providing search to local models.

I'm curious what setups folks use to provide this functionality.

Since the quantized 24B parameter Gemma model came out, I've had good luck with tool calling on a 4070 Ti Super.

Successful tool calling is what finally made the local experience useful.

I should note this is for the general and not coding specific context.

gardnr an hour ago | parent | next [-]

It has a JSON mode that you need to enable in settings and then you can create a simple python script to interact with it or have the agent use `curl` and `jq` to interact with it.

It's at the bottom of this page: https://docs.searxng.org/admin/settings/settings_search.html

drnick1 an hour ago | parent | prev [-]

I am also interested in what a full local AI stack with web search and other tools looks like. As far as I can tell, SearX does not embed an MCP server, so it can't be directly called from llama-server for example. Open WebUI does have an integration for SearX and other providers, but the results I obtained weren't particularly impressive.

artooro 3 hours ago | parent | prev | next [-]

It works well if you connect it the Brave Search API, but using it a scraper is fairly unreliable. Google stopped working a few days ago.

fishgoesblub 2 hours ago | parent | prev | next [-]

I've been using SearXNG for a few years now, however I've been trying out Degoog as a SearXNG alternative since I've had issues with engines constantly failing or being slow since day 1 of using SearXNG, but Degoog has worse results with the same engines. It's a shame since I'm having to pick between slower but better results, or very fast but worse results.

dexterdog 3 hours ago | parent | prev | next [-]

I've been self hosting this as my default engine across all of my searches for a few years now. I can't recommend it more highly.

viviansolide 3 hours ago | parent | next [-]

Same experience

ProofHouse 3 hours ago | parent | prev [-]

I’ll have to try, I’ve only recently learned Exa pricing is a bit crazy (especially on searches where you source 30-40 sources)I just used it be default and then was like oh damn when I got hit

ManWith2Plans 3 hours ago | parent | prev | next [-]

I've been using this for some projects. It's exceptional and I recommend it highly.

I actually included a recipe to deploy it to kubernetes in typekro, my TypeScript infrastructure-as-code project for kubernetes: https://typekro.run/api/searxng/

arikrahman 3 hours ago | parent | prev | next [-]

I have used SearXNG hosts like https://searx.be/ but stick with Brave search for the most part. Are there other good hosts people tend to use?

vimredo 3 hours ago | parent [-]

Personally, I self-host it myself. All the hosts I tried either errored often, or gave search results that were complete garbage.

lucasrufkahr 2 hours ago | parent | prev | next [-]

Yeah, I find that searx results are way more relevant to what I’m actually looking for than a single engine. There’s so much manipulation going on that if you don’t aggregate multiple engines, it’s near impossible to get what you want.

rcarmo 2 hours ago | parent | prev | next [-]

Years of regular use here, has been great even before I started using it as an agent tool.

another_twist 3 hours ago | parent | prev | next [-]

Been a fan of searX for a while. Not sure if this is the same thing but there were plenty of hosted versions too.

salmonik 3 hours ago | parent | prev | next [-]

I prefer 4get.

noobcoder 2 hours ago | parent | prev [-]

how do i configure which specific search engines SearXNG pulls its results from? Can we extend it to onyl search Stack Overflow and GitHub