← All examples
01 / Python
Web scraping with Python
Fetch a public page with a timeout, an environment-based proxy, and explicit error handling.
You will build
A small, production-friendly requests flow
ScrapingRetries
Python
import os
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
key = os.environ["PROXYAPI_KEY"]
proxy = f"https://{key}-country-se-type-isp:@proxy.proxyapi.se:443"
session = requests.Session()
session.proxies.update({"http": proxy, "https": proxy})
session.mount("https://", HTTPAdapter(max_retries=Retry(
total=3,
backoff_factor=0.5,
status_forcelist=[429, 500, 502, 503, 504],
)))
response = session.get("https://example.com", timeout=20)
response.raise_for_status()
print(response.status_code, len(response.content))Before production
- ✓Check the target site’s terms, robots policy, and applicable law before collection.
- ✓Log status and duration, but never print the API key.