How to Make a GET Request in Python with Requests

This guide shows you how to make a GET request in Python using the requests library, the de facto standard for HTTP in Python. You’ll learn to fetch data from a URL, handle query parameters, inspect the response, and deal with common errors. By the end, you’ll be able to pull JSON from APIs, download web pages, and integrate HTTP calls into your scripts.

What You’ll Need

• Python 3.6 or newer installed on your system.
• The requests library installed. If you don’t have it, run pip install requests in your terminal or command prompt.
• A code editor or Python IDE (like VS Code, PyCharm, or even a simple text editor).
• An internet connection to make real requests.

Step-by-Step

1. Import the requests library
   At the top of your Python file, add import requests. This gives you access to all the functions you need.
2. Make a simple GET request
   Use the get() function with the URL you want to fetch. For example:
   response = requests.get('https://api.github.com')
   This sends a GET request to that URL and stores the server’s response in the response object.
3. Check the response status code
   Before using the data, verify the request succeeded. The status code 200 means OK. Use response.status_code to see it. For example:
   if response.status_code == 200:
print('Success!')
else:
print('Failed with status:', response.status_code)
4. Read the response content
   The response body is available in several formats:
   • response.text – raw text (e.g., HTML or plain text).
   • response.content – bytes (useful for binary data like images).
   • response.json() – parses the body as JSON and returns a Python dictionary/list. Use this when the API returns JSON.

   Example: data = response.json()

5. Add query parameters
   If the URL requires parameters (like search terms or pagination), pass them via the params argument. This is cleaner than building the URL string manually.
   params = {'q': 'python', 'page': 2}
response = requests.get('https://api.github.com/search/repositories', params=params)
   The library will encode them properly, producing a URL like ?q=python&page=2.
6. Set custom headers (optional)
   Some APIs require headers like User-Agent or an API key. Pass them with the headers parameter.
   headers = {'User-Agent': 'MyApp/1.0'}
response = requests.get('https://api.example.com/data', headers=headers)
7. Handle timeouts
   Always set a timeout to avoid hanging forever. Use the timeout parameter (in seconds).
   response = requests.get('https://api.example.com', timeout=5)
   This raises a Timeout exception if the server doesn’t respond within 5 seconds.
8. Use the response in your code
   Once you have the data, you can process it. For JSON, you might loop through items, extract fields, or save to a file.

Common Mistakes

• Forgetting to check status_code – Assuming the request succeeded can lead to errors when the API returns 404 or 500. Always check.
• Using response.json() on non-JSON responses – This raises ValueError. Check the content-type or wrap it in a try-except.
• Not encoding query parameters manually – If you build URLs with spaces or special characters, they break. Use the params argument instead.
• Ignoring timeouts – Without a timeout, your script can hang indefinitely if the server is slow or down.
• Not closing the response – While requests handles this automatically in most cases, using a context manager (with requests.get(...) as response:) ensures proper cleanup.

Tips

• Use response.raise_for_status() – This method raises an HTTPError for bad status codes (4xx or 5xx). It’s a concise way to handle errors.
• Inspect the URL – response.url shows the final URL after redirects and parameter encoding. Useful for debugging.
• Use sessions for multiple requests – If you’re making several requests to the same host, use requests.Session(). It reuses the underlying TCP connection, which is faster.
• Set a custom User-Agent – Some servers block default Python user agents. A simple header like User-Agent: MyApp can avoid 403 errors.
• Handle redirects explicitly – By default, requests follows redirects. If you don’t want that, set allow_redirects=False.

Troubleshooting

If your request fails, here’s what to do:

• ConnectionError – This means the URL is unreachable. Check your internet connection, verify the URL is correct, and ensure you’re not behind a firewall that blocks outbound requests.
• Timeout – The server didn’t respond in time. Increase the timeout value, or check if the server is overloaded. Also, consider using a retry mechanism.
• HTTP 403 Forbidden – The server understands the request but refuses to authorize it. This often happens when the server blocks default user agents or requires authentication. Add a proper User-Agent header or check if you need an API key.
• HTTP 404 Not Found – The URL is wrong. Double-check the path, including case sensitivity and trailing slashes.
• JSONDecodeError – The response isn’t valid JSON. Print response.text to see what you actually received. It might be an error message in HTML.
• SSL certificate errors – If you get an SSL error, it’s usually because the server has an invalid certificate. For testing, you can use verify=False, but never do this in production – it’s a security risk.

Frequently Asked Questions

Do I need to install requests separately?

Yes, requests is not part of the standard library. Install it with pip install requests. If you’re using a virtual environment, activate it first.

How do I send query parameters that contain spaces or special characters?

Use the params argument. The requests library automatically URL-encodes them. For example, {'q': 'python programming'} becomes q=python+programming.

What’s the difference between response.text and response.content?

response.text is a string decoded using the response’s encoding (usually UTF-8). response.content is raw bytes. Use .text for text data and .content for binary files like images or PDFs.

Can I make a GET request with authentication?

Yes, you can pass credentials using the auth parameter, e.g., requests.get(url, auth=('username', 'password')). For token-based auth, use headers: headers={'Authorization': 'Bearer YOUR_TOKEN'}.

How do I handle a response that is not JSON?

Check the content-type header first: response.headers.get('content-type'). If it’s not JSON, use response.text or response.content instead. If you expect JSON but get something else, the server might be returning an error page.

Is there a way to avoid blocking when making many requests?

Yes, use a Session object and set appropriate headers. Also, add delays between requests (time.sleep(1)) to be polite to the server. Some APIs have rate limits – check their documentation.

Related

• Choosing a Samsung Galaxy Tab S12 Case
• Make Spotify Playlists Public Automatically
• Fix Android Phone Not Receiving Emails