Skip to content

Web crawler

NteniBot.
What it reads, and how to stop it.

NteniBot is the web crawler run by Inkal OÜ. It reads public pages on company websites, such as About, Team, Leadership and Contact, to keep a directory of businesses and their published business contacts up to date.

How to recognise it

It always says who it is.

Every request carries the user agent NteniBot/1.0 (+https://nteni.com/bot).

What it collects. Company name, a short description of what the company does, and business contact details that the company itself publishes: names and job titles of staff listed on the site, and business email addresses shown on its pages. It does not log in, fill in forms, solve CAPTCHAs or read pages behind a paywall, and it does not collect sensitive personal information.

Obeys robots.txt

Including rules written for NteniBot and any Crawl-delay you set.

Reads very little

A handful of pages per site per visit, one at a time, at least two seconds apart.

Stops when asked

A 401 or 403 answer stops visits to the site. On 429 or 503 it waits as long as the site asks.

Visits rarely

It returns to a site at most every few weeks.

Block or remove

Your site, your choice.

To block NteniBot, add this to your robots.txt:

User-agent: NteniBot
Disallow: /

To remove your company or your own business contact details from nteni, email privacy@nteni.com. We act on requests within 30 days and keep the address on a suppression list so it is not collected again.

Questions about NteniBot: crawler@nteni.com. Read the data rights page and the privacy policy for how we handle personal data.