(Python Basic Tutorial XIII) Examples of using httplib2-HTTP GET and POST in Python

Learn to use Python httplib2 Modular.Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, and hypermedia information systems.HTTP is the basis for data communication on the World Wide Web.

The Python httplib2 module provides methods for accessing Web resources over HTTP.It supports many functions, such as HTTP and HTTPS, authentication, caching, redirection, and compression.

$ service nginx status

* nginx is running

We run the nginx Web server on the local host.Some of our examples will connect to PHP scripts on a locally running nginx server.

Catalog Check the httplib2 library version Use httplib2 to read web pages Send HTTP HEAD request Send HTTP GET request Send HTTP POST request Send User Agent Information Add username/password to request

Check the httplib2 library version

The first program prints the version of the library, its copyright and document string.

#!/usr/bin/python3

import httplib2

print(httplib2.__version__)

print(httplib2.__copyright__)

print(httplib2.__doc__)

In the version httplib2 library given by httplib2. u version_, httplib2. u copyright_ gives its copyright and its document string httplib2. u doc_.

$ ./version.py

0.8

Copyright 2006, Joe Gregorio

httplib2

A caching http interface that supports ETags and gzip

to conserve bandwidth.

Requires Python 3.0 or later

Changelog:

2009-05-28, Pilgrim: ported to Python 3

2007-08-18, Rick: Modified so it's able to use a socks proxy if needed.

This is the sample output of the example.

Use httplib2 to read web pages

In the following example, we show how to start with www.something.com Web site to get HTML content.

#!/usr/bin/python3

import httplib2

http = httplib2.Http()

content = http.request("[http://www.something.com](http://www.something.com)")[1]

print(content.decode())

Use Create an HTTP Client httplib2.HTTP().Use the request() method to create a new HTTP request.By default, it is a GET request.The return value is a tuple of response and content.

$ ./get_content.py

<html><head><title>Something.</title></head>

<body>Something.</body>

</html>

This is the output of the example.

Strip HTML Tags

The following program takes a small web page and strips its HTML tags.

#!/usr/bin/python3

import httplib2

import re

http = httplib2.Http()

content = http.request("[http://www.something.com](http://www.something.com)")[1]

stripped = re.sub('<[^<]+?>', '', content.decode())

print(stripped)

A simple regular expression is used to strip HTML tags.Please note that we are stripping data and we have not cleaned it up.(These are two things.)

$ ./strip_tags.py

Something.

Something.

The script will print the title and content of the page.

Check response status

The response object contains the properties that status provides the response status code.

#!/usr/bin/python3

import httplib2

http = httplib2.Http()

resp = http.request("[http://www.something.com](http://www.something.com)")[0]

print(resp.status)

resp = http.request("[http://www.something.com/news/](http://www.something.com/news/)")[0]

print(resp.status)

We execute two HTTP requests using the request() method and check the status returned.

$ ./get_status.py

200

404

200 is the standard response to a successful HTTP request, while 404 indicates that the requested resource was not found.

Send HTTP HEAD request

The HTTP HEAD method retrieves the document title.The header consists of fields, including date, server, content type, or last modified time.

#!/usr/bin/python3

import httplib2

http = httplib2.Http()

resp = http.request("[http://www.something.com](http://www.something.com)", "HEAD")[0]

print("Server: " + resp['server'])

print("Last modified: " + resp['last-modified'])

print("Content type: " + resp['content-type'])

print("Content length: " + resp['content-length'])

This sample print server, last modified time, content type, and content length of www.something.com pages.

$ ./do_head.py

Server: Apache/2.4.12 (FreeBSD) OpenSSL/1.0.1l-freebsd mod_fastcgi/mod_fastcgi-SNAP-0910052141

Last modified: Mon, 25 Oct 1999 15:36:02 GMT

Content type: text/html

Content length: 72

This is the output of the program.From the output, we can see that the page was delivered by the Apache Web server hosted by FreeBSD.The document was last modified in 1999.A Web page is an HTML document that is 72 bytes long.

Send HTTP GET request

The HTTP GET method requests a representation of the specified resource.For this example, we will also use the greet.php script:

<?php

echo "Hello " . htmlspecialchars($_GET['name']);

?>

In the / usr/share/nginx/html/directory, we have this greet.php file.The script returns the value of the name variable, which is retrieved from the client.

The htmlspecialchars() function converts special characters to HTML entities; for example, &to&amp.

#!/usr/bin/python3

import httplib2

http = httplib2.Http()

content = http.request("[http://localhost/greet.php?name=Peter](http://localhost/greet.php?name=Peter)",

method="GET")[1]

print(content.decode())

The script sends variables with values to the PHP script on the server.This variable is specified directly in the URL.

$ ./mget.py

Hello Peter

//This is the output of the example.

$ tail -1 /var/log/nginx/access.log

127.0.0.1 - - [21/Aug/2016:17:32:31 +0200] "GET /greet.php?name=Peter HTTP/1.1" 200 42 "-"

"Python-httplib2/0.8 (gzip)"

We checked the nginx access log.

Send HTTP POST request

The POST request method requests the Web server to accept and store the data contained in the body of the request message.It is often used when uploading files or submitting complete Web forms.

<?php

echo "Hello " . htmlspecialchars($_POST['name']);

?>

On the local Web server, we have this target.php file.It simply prints the posted value back to the customer.

#!/usr/bin/python3

import httplib2

import urllib

http = httplib2.Http()

body = {'name': 'Peter'}

content = http.request("[http://localhost/target.php](http://localhost/target.php)",

method="POST",

headers={'Content-type': 'application/x-www-form-urlencoded'},

body=urllib.parse.urlencode(body) )[1]

print(content.decode())

The script sends a request for a key whose name has a Peter value.The data is encoded using the urllib.parse.urlencode() method and sent in the body of the request.

$ ./mpost.py

Hello Peter

This is the output of the mpost.py script.

$ tail -1 /var/log/nginx/access.log

127.0.0.1 - - [23/Aug/2016:12:21:07 +0200] "POST /target.php HTTP/1.1"

200 37 "-" "Python-httplib2/0.8 (gzip)"

When using the POST method, the value is not sent in the request URL.

Send User Agent Information

In this section, we specify the name of the user agent.

<?php

echo $_SERVER['HTTP_USER_AGENT'];

?>

In the nginx document root directory, we have the agent.php file.It returns the name of the user agent.

#!/usr/bin/python3

import httplib2

http = httplib2.Http()

content = http.request("[http://localhost/agent.php](http://localhost/agent.php)", method="GET",

headers={'user-agent': 'Python script'})[1]

print(content.decode())

The script creates a simple GET request agent.php from the script.In the headers dictionary, we specify the user agent.The PHP script reads this content and returns it to the client.

$ ./user_agent.py

Python script

The server responded with the proxy name we sent with the request.

Add username/password to request

The client's add_credentials() method sets the name and password used for the realm.The security realm is a mechanism for protecting Web application resources.

$ sudo apt-get install apache2-utils

$ sudo htpasswd -c /etc/nginx/.htpasswd user7

New password:

Re-type new password:

Adding password for user user7

We use the htpasswd tool to create user names and passwords for basic HTTP authentication.

location /secure {

auth_basic "Restricted Area";

auth_basic_user_file /etc/nginx/.htpasswd;

}

In the nginx/etc/nginx/sites-available/default configuration file, we create a secure page.The domain is named "restricted zone".

<!DOCTYPE html>

<html lang="en">

<head>

<title>Secure page</title>

</head>

<body>

<p>

This is a secure page.

</p>

</body>

</html>

In the / usr/share/nginx/html/security directory, we have the HTML file above.

#!/usr/bin/python3

import httplib2

user = 'user7'

passwd = '7user'

http = httplib2.Http()

http.add_credentials(user, passwd)

content = http.request("[http://localhost/secure/](http://localhost/secure/)")[1]

print(content.decode())

The script connects to a secure Web page; it provides the user name and password required to access the page.

$ ./credentials.py

<!DOCTYPE html>

<html lang="en">

<head>

<title>Secure page</title>

</head>

<body>

<p>

This is a secure page.

</p>

</body>

</html>

With the correct credentials, the script returns to the protected page.

In this tutorial, we explored the Python httplib2 module.

  1. Python Basic Tutorial
  2. Configuring the Python environment in SublimeEditor
  3. Add comments to Python code
  4. Use of variables in Python
  5. Data types in Python
  6. Keyword in Python
  7. Python string operations
  8. list operation in Python
  9. Tuple operation in Python
  10. Pythonmax() and min() - Find the maximum and minimum values in a list or array
  11. Python finds the largest N (first N) or smallest N items
  12. Python Read and Write CSV Files
  13. Examples of using httplib2-HTTPGET and POST in Python
  14. Python boxes tuple as a variable or parameter
  15. Python Out-of-Box Tuple - Too many values cannot be uncompressed
  16. Pythonmultidict example - Mapping a single key to multiple values in a dictionary
  17. PythonOrderedDict - Ordered Dictionary
  18. Python Dictionary Intersection - Compare Two Dictionaries
  19. Python Priority Queue Example Author: Distributed programming Source: https://zthinker.com/ If you like this, long press QR code and focus on Distributed Programming .

Tags: Programming Python PHP Nginx Web Server

Posted on Sun, 10 May 2020 18:23:24 -0700 by shamsuljewel