DeepSearch on XtoolX is a web front end for the open-source Indiblog/deepsearch engine. Search your own index of publicly reachable sites, open directories, FTP listings, datasets, and archived URLs.
How it works
The XtoolX page talks to a DeepSearch API running on the same server (via a safe proxy).
Use Search index to query pages your crawler has already indexed.
Use Wayback discover to pull matching URLs from the Internet Archive without waiting for a crawl.
Use Domain discover to enumerate public hostnames from certificate transparency and DNS sources.
Why it matters
Explore niche and self-hosted content that mainstream search engines rarely surface.
Keep discovery and indexing under your control instead of relying on third-party SaaS crawlers.
Pair with Domain Lookup on XtoolX for certificate-log hostnames versus full-text search over crawled pages.
FAQ
Do I need to install anything?
Visitors use the XtoolX UI in the browser. The server operator must run the DeepSearch API (see deploy/deepsearch.md) and point DEEPSEARCH_API_URL at it.
Why are there no search results?
DeepSearch only searches what your instance has crawled and indexed. Seed URLs and run a crawl on the backend before expecting hits.
Can I start a crawl from this page?
No. Public crawl controls are disabled for safety. Run crawls on the server with the DeepSearch CLI or admin processes.
Is this legal?
DeepSearch is meant for publicly accessible resources only. You are responsible for complying with laws and site terms where you operate the crawler.
Open the interactive tool on this page after JavaScript loads, or contact us if something is broken.