When it comes to making HTTP requests, for developers the goto library is requests. But there is another library that has already a large fanbase and popularity is growing, it’s httpx.

Both serve the purpose of sending HTTP requests, but they have subtle differences that can impact your choice depending on your specific needs.

In this blog post, we will explore the features of Python’s requests and httpx library.

How to install

requests

pip install requests

httpx

Using pip

pip install httpx

Also, as a cli

pip install 'httpx[cli]'

To check available commands

httpx --help

Synchronous and Asynchronous Support

requests

Primarily synchronous, meaning that each request blocks the execution until a response is received.

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

if response.status_code == 200:
    print("Response Content:")
    print(response.text)
else:
    print(f"Error: {response.status_code} - {response.text}")

httpx

Supports both synchronous and asynchronous request handling, making it more versatile for high-performance applications.

import httpx
import asyncio

async def get_pots(url):
    async with httpx.AsyncClient() as client:
        try:
            response = await client.get(url)

            if response.status_code == 200:
                print("Response Content:")
                print(response.text)
            else:
                print(f"Error: {response.status_code} - {response.text}")
        except httpx.RequestError as e:
            print(f"An error occurred: {e}")

url = "https://jsonplaceholder.typicode.com/posts/1"

# Run the asynchronous GET request
asyncio.run(get_pots(url))

HTTP/2 Support

requests

Primarily focuses on HTTP/1.1.

httpx

Supports the more modern HTTP/2 protocol. This can result in improved performance, especially for applications with many concurrent requests.

import httpx
import asyncio

async def http2_get(url):
    async with httpx.AsyncClient(http2=True) as client:
        try:
            response = await client.get(url, http_version="HTTP/2")

            if response.status_code == 200:
                print("Response Content:")
                print(response.text)
            else:
                print(f"Error: {response.status_code} - {response.text}")

        except httpx.RequestError as e:
            print(f"An error occurred: {e}")

url = "https://http2.golang.org/reqinfo"
# Run the asynchronous HTTP/2 request
asyncio.run(http2_get(url))

Connection Pooling

requests

Supports connection pooling.

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"

# Create a session for connection pooling
session = requests.Session()

# Make multiple GET requests using the same session (connection pooling)
for _ in range(5):
    response = session.get(url)
    
    if response.status_code == 200:
        print("Response Content:")
        print(response.text)
    else:
        print(f"Error: {response.status_code} - {response.text}")

# Close the session to release resources
session.close()

httpx

Takes it a step further by using a connection pool per remote host by default. This can lead to more efficient resource utilization and reduced latency in certain scenarios.

import httpx

url = "https://jsonplaceholder.typicode.com/posts/1"

# Create an HTTPX client with connection pooling
client = httpx.Client(keep_alive=True)

# Make multiple GET requests using the same client (connection pooling)
for _ in range(5):
    try:
        response = client.get(url)

        if response.status_code == 200:
            print("Response Content:")
            print(response.text)
        else:
            print(f"Error: {response.status_code} - {response.text}")

    except httpx.RequestError as e:
        # Handle exceptions, such as connection errors
        print(f"An error occurred: {e}")

# Close the client to release resources
client.close()

Timeouts and Retries

requests

Provides basic timeout and retry options.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
from requests.packages.urllib3.util.timeout import Timeout

url = "https://jsonplaceholder.typicode.com/posts/1"

retry_strategy = Retry(total=3, backoff_factor=1, status_forcelist=[500, 502, 503, 504])
timeout = Timeout(connect=2, read=5)

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount("http://", adapter)
session.mount("https://", adapter)

try:
    response = session.get(url, timeout=timeout)

    if response.status_code == 200:
        print("Response Content:")
        print(response.text)
    else:
        print(f"Error: {response.status_code} - {response.text}")

except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

session.close()

httpx

Offers more fine-grained control over timeouts and automatic retries, enhancing the robustness of your applications.

import httpx
import asyncio

url = "https://jsonplaceholder.typicode.com/posts/1"

async def request_with_retry_and_timeout(url):
    async with httpx.AsyncClient() as client:
        try:
            response = await client.get(url, timeout=(2, 5), retry=3)

            if response.status_code == 200:
                print("Response Content:")
                print(response.text)
            else:
                print(f"Error: {response.status_code} - {response.text}")

        except httpx.RequestError as e:
            print(f"An error occurred: {e}")

# Run the asynchronous request with retry and timeout
asyncio.run(request_with_retry_and_timeout(url))

Personal Opinion

Choosing between requests and httpx depends on your project’s requirements and your personal preferences. If you value simplicity and a mature, stable library, requests might be the better choice.

On the other hand, if you are working on a modern application that can benefit from asynchronous requests, HTTP/2 support, and more advanced features, httpx could be the way to go.