5. What is a User Agent?

User Agent is a string that a client application passes to the server in an HTTP request to identify itself. For example, what browser or bot it is, or what operating system it runs on. Sometimes version or contact information is passed as well. For a webmaster, this is one of the first and easiest signals about who is accessing the site. But it’s dangerous to rely on it alone: the string is easily spoofed, and modern browsers reduce its detail for the sake of privacy.

What the standard says

The «User Agent» field is a standard HTTP header. The HTTP specification describes it as a way to tell the server information about the client: why it is done, when there are compatibility issues, and as a statistical tool. The RFC records a recommendation: the agent should send this field unless it is explicitly disabled.

Browsers have historically included a combination of «name/version», rendering engine, and platform details in the User Agent. Over time, the string has evolved into a long set of labels, partly for historical compatibility reasons, partly for the desire of services to recognize devices and user behavior.

What User Agents are there

• Classic browsers. Chrome, Firefox, Safari and their derivatives. Strings contain engine, platform and version data. Modern browser versions are gradually reducing this data in favor of Client Hints.
• Mobile browsers and apps-embedded WebView. Often contain device model and operating system information.
• Search engine robots and crawlers. Official bots (Googlebot, Bingbot and others) usually provide their name and a rules page. They are useful to keep track of. Sometimes they are given extended access and sometimes they are restricted.
• Libraries and query tools. Curl, Wget, Python-requests and HTTP clients often send simple User Agents or leave the field blank by default.
• Headless-browsers and autotests. Puppeteer, Playwright, PhantomJS and similar tools sometimes label themselves explicitly, sometimes masquerading as regular browsers.
• Proxies, scrapers and custom clients. Everything from neatly designed contact strings to deliberate spoofing of popular browsers can be found here.

Modern Changes

In recent years, browsers have been moving towards reducing the details in the traditional User Agent string to reduce the possibility of tracking by device attributes. Instead, Client Hints engines are being introduced, this is a set of special headers and interfaces through which the server can request specific information from the client as needed.

Major engines have plans and practical steps to «eliminate» User Agent. In general, site owners are advised to migrate to Client Hints if accurate client information is needed.

Why a site needs to know User Agent

User Agent is used for analytics, adaptive content delivery, and for interacting with search engine robots. But there are a few important caveats:

• The string is easy to spoof. Any HTTP client can send any text in this field; so relying on it as a reliable attribute is risky.
• Compatibility. Many sites old-fashionedly «tailor» content to the browser string. This leads to errors and unnecessary support for outdated combinations.
• Privacy. A granular User Agent provides additional attributes to the browser fingerprint, so platforms are looking to reduce its informativeness.

In 0DETECT anti-detection browser, you can customize User Agent settings. And you can use both the already filled in variant and set your own.