Python Full Stack Development-Common Module Learning

Python Full Stack Development-Common Module Learning

Module Introduction

  • Time & datetime module
  • random
  • os
  • sys
  • shutil
  • shelve
  • xml processing
  • pyyaml processing
  • configparser
  • hashlib
  • re regular expression

1. Introduction of Modules

1) Definition:

Module: Used to logically organize Python code (variables, functions, classes, logic: to implement a function), the essence is the python file at the end of. py (file name:, module name is: test)

Package: Used to logically organize modules, essentially a directory (must have a file)

2) Import method:

  import module_name

  import module_name, module2_name

from module_gavin import means importing all methods from module_gavin. It is equivalent to copying all the code under this module to the current main program. This method is not recommended because it may be overwritten by renaming some functions in the main program. Therefore, this method is not recommended.

from module_gavin import logger is a logger method imported only under this module.

from module_gavin import logger as logger_gavin aliases the logger method under this module: logger_gavin, which is called directly by the main program when it is called.

3) the essence of import (path search and search path):

The essence of the import module is to interpret the python file once, and then encapsulate the result of the interpretation, assigning it to a variable called "module name". Therefore, when referring to the following program, the encapsulated module name needs to be added in front of it.

The essence of importing a package is to execute the file under that package

FormXXX import XXX is essentially equivalent to copying all or corresponding code under the module to the current main program and explaining it once. Referenced variables or functions can be used directly in the main program without the need to add module names.

4) Import optimization:

from module_gavin import logger Using this method, the logger function is directly copied and explained, omitting the process of finding the module name in import module_gavin method. If the logger method under this module is called in a large range of programs, the module_gavin module needs to be found first for each call. Then the following method can be invoked. By using the method from XXX import XXX, the process of finding modules can be omitted and the speed is greatly improved.

5) Module classification:

a, standard library, or built-in modules

b, open source module, or third-party module

c. Custom Module

2. Time & datetime module

In Python, there are usually several ways to represent time: 1) timestamp 2) formatted time string 3) tuple (struct_time) with nine elements. Because Python's time module implements calling C libraries, platforms may differ.

UTC (Coordinated Universal Time) is Greenwich Astronomical Time, World Standard Time. UTC+8 in China. DST (Daylight Saving Time) is daylight saving time.

Time stamp: Typically, a time stamp represents an offset calculated in seconds starting at 00:00 on January 1, 1970. We run "type" (time. time()) and return the float type. The main functions that return the timestamp are time(), clock(), and so on.

Strct_time: There are nine elements in the struct_time tuple. The main functions returning struct_time are gmtime(), localtime(), strptime(). Here are a few elements in this way tuple:

import time
# print(time.clock()) #Return to the processor time, 3.3 has been abandoned, changed to time.process_time() to measure the processor operation time, excluding sleep time, instability, mac can not be measured.
# print(time.altzone)  #The time difference between return and utc time is calculated in seconds\
# print(time.asctime()) #Return the time format "Fri Aug 19 11:14:16 2016".
# print(time.localtime()) #Returns the struct time object format for local time
# print(time.gmtime(time.time()-800000)) #struc time object format returning utc time
# print(time.asctime(time.localtime())) #Return the time format "Fri Aug 19 11:14:16 2016".
#print(time.ctime()) #Return to Fri Aug 19:12:38:29 2016, ibid.
# Date string to timestamp
# string_2_struct = time.strptime("2016/05/22","%Y/%m/%d") #Converting date strings to struct time object format
# print(string_2_struct)
# #
# struct_2_stamp = time.mktime(string_2_struct) #Converting a struct time object to a timestamp
# print(struct_2_stamp)
#Converting the timestamp to string format
# print(time.gmtime(time.time()-86640)) #Converting utc timestamp to struct_time format
# print(time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime()) ) #Converts the utc struct_time format to the specified string format
#Time addition and subtraction
import datetime
# print( #Return to 2016-08-19 12:47:03.94 1925
#print( )  # Time stamp directly converted to date format 2016-08-19
# print( )
# print( + datetime.timedelta(3)) #Current time + 3 days
# print( + datetime.timedelta(-3)) #Current time - 3 days
# print( + datetime.timedelta(hours=3)) #Current time + 3 hours
# print( + datetime.timedelta(minutes=30)) #Current time + 30
# c_time  =
# print(c_time.replace(minute=3,hour=2)) #Time replacement

Format Reference

% a locale simplified week name    
% A Full Local Weekly Name    
% b Locally simplified month names    
% B Local full month name    
% c Local corresponding date and time representation    
% d The day of the month (01 - 31)    
% H ours of the day (24 hours, 00 - 23)    
% I Hours (12 hours, 01 - 12)    
% j Days of the year (001 - 366)    
% m month (01 - 12)    
% M minutes (00 - 59)    
% p local am or pm corresponding to 1    
% S second (01 - 61) 2    
% The number of weeks in a year. Sunday 00 - 53 is the beginning of a week. All days before the first Sunday are in week 0. Three    
% What day of the w eek (0 - 6, 0 is Sunday) Wednesday    
% W and% U are basically the same, but% W begins on Monday as a week.    
% x Local Corresponding Date    
% X Local Corresponding Time    
% y Remove the year of the century (00 - 99)    
% Y Complete Year    
% Name of Z time zone (if no null character exists)    

Time Relation Conversion

3. random module

#!/usr/bin/env python
#_*_encoding: utf-8_*_
import random
print (random.random())  #0.6445010863311293  
#random.random() is used to generate a number of random characters from 0 to 1: 0 <= n < 1.0
print (random.randint(1,7)) #4
#The function prototype of random.randint() is random.randint(a, b), which is used to generate an integer in a specified range.
# Where parameter a is the lower limit and parameter b is the upper limit, the random number n: a <= n <= b is generated.
print (random.randrange(1,10)) #5
#The function prototype of random.randrange is random.randrange([start], stop[, step]).
# Gets a random number from a set incremented by a specified Radix within a specified range. For example: random.randrange(10, 100, 2),
# The result is equivalent to obtaining a random number from the sequence [10, 12, 14, 16,... 96, 98].
# Randrange (10, 100, 2) is equivalent to random.choice(range(10, 100, 2).
print(random.choice('liukuni')) #i
#random.choice takes a random element from the sequence.
# Its function prototype is random.choice(sequence). The parameter sequence represents an ordered type.
# Here's a note: sequence is not a specific type in python, but refers to a series of types in general.
# list, tuple, and strings all belong to sequence. For sequence, you can see the chapter on python manual data model.
# Here are some examples of using choice:
print(random.choice("Study Python"))#learn
print(random.choice(["JGood","is","a","handsome","boy"]))  #List
print(random.choice(("Tuple","List","Dict")))   #List
print(random.sample([1,2,3,4,5],3))    #[1, 2, 5]
#The function prototype of random.sample is random.sample(sequence, k), which randomly retrieves fragments of specified length from a specified sequence. The sample function does not modify the original ordered column.

Practical application:

#!/usr/bin/env python
# encoding: utf-8
import random
import string
#Random integers:
print( random.randint(0,99))  #70
#Randomly select even numbers between 0 and 100:
print(random.randrange(0, 101, 2)) #4
#Random Floating Point:
print( random.random()) #0.2746445568079129
print(random.uniform(1, 10)) #9.887001463194844
#Random Characters:
print(random.choice('abcdefg&#%^*f')) #f
#Select a specific number of characters from multiple characters:
print(random.sample('abcdefghij',3)) #['f', 'h', 'd']
#Random selection of strings:
print( random.choice ( ['apple', 'pear', 'peach', 'orange', 'lemon'] )) #apple
#Shuffle the cards#
items = [1,2,3,4,5,6,7]
print(items) #[1, 2, 3, 4, 5, 6, 7]
print(items) #[1, 4, 7, 2, 5, 3, 6]

Generate random validation code:

import random
checkcode = ''
for i in range(4):
    current = random.randrange(0,4)
    if current != i:
        temp = chr(random.randint(65,90))
        temp = random.randint(0,9)
    checkcode += str(temp)
print (checkcode)

4. os module

Provide an interface for calling the operating system

os.getcwd() gets the current working directory, which is the directory path for the current python script to work on
 os.chdir("dirname") changes the current script working directory; equivalent to cd under shell
 os.curdir returns the current directory: ('.')
os.pardir gets the parent directory string name of the current directory: ('.')
os.makedirs('dirname1/dirname2') can generate multiple recursive directories
 os.removedirs('dirname1') If the directory is empty, delete it and recurse to the higher level directory. If it is also empty, delete it, and so on.
os.mkdir('dirname') generates a single-level directory; equivalent to mkdir dirname in the shell
 os.rmdir('dirname') deletes a single-level empty directory and cannot be deleted if the directory is not empty, causing an error; equivalent to rmdir dirname in the shell
 os.listdir('dirname') lists all files and subdirectories in the specified directory, including hidden files, and prints them in a list format
 os.remove() deletes a file
 os.rename("oldname","newname") renames files / directories
 os.stat('path/filename') Gets file/directory information
 os.sep outputs the operating system-specific path separator, under win is "\" and under Linux is "/"
os.linesep outputs the line terminator used by the current platform, with "t\n" under win and "n" under Linux.
os.pathsep outputs strings for splitting file paths
 The output string indicates the current platform to be used. Win - >'nt'; Linux - >'posix'
os.system("bash command") runs the shell command and displays it directly
 os.environ Gets System Environment Variables
 os.path.abspath(path) returns the absolute path normalized by path
 os.path.split(path) divides path into directory and file name binaries to return
 os.path.dirname(path) returns the directory of path. It's actually the first element of os.path.split(path)
os.path.basename(path) returns the last file name of path. How path ends with / or, then null values are returned. That is, the second element of os.path.split(path)
os.path.exists(path) Returns True if path exists; False if path does not exist
 os.path.isabs(path) If path is an absolute path, return True
 os.path.isfile(path) If path is an existing file, return True. Otherwise return to False
 os.path.isdir(path) returns True if path is an existing directory. Otherwise return to False
 os.path.join(path1[, path2 [,...]]) returns after combining multiple paths, and the parameters before the first absolute path are ignored
 os.path.getatime(path) returns the last access time of the file or directory to which path refers
 os.path.getmtime(path) returns the last modification time of the file or directory to which path refers

5. sys module

sys.argv           Command line parameters List,The first element is the path of the program itself.
sys.exit(n)        Exit procedure, normal exit time exit(0)
sys.version        Obtain Python Version information of interpreter
sys.maxint         Maximal Int value
sys.path           Returns the search path of the module, which is used for initialization PYTHONPATH Values of environmental variables
sys.platform       Returns the name of the operating system platform
val = sys.stdin.readline()[:-1]

6. shutil module

Direct reference

7. Sheve module

The shell module is a simple k,v module that persists memory data through files and can persist any python data format that pickle can support.

import shelve
d ='shelve_test') #Open a file
class Test(object):
    def __init__(self,n):
        self.n = n
t = Test(123) 
t2 = Test(123334)
name = ["alex","rain","test"]
d["test"] = name #Persistence List
d["t1"] = t      #Persistence class
d["t2"] = t2

8. xml processing module

XML is a protocol for data exchange between different languages or programs, similar to json, but JSON is easier to use. However, in ancient times, in the dark age before JSON was born, people could only choose to use xml. So far, many traditional companies, such as many systems in the financial industry, still mainly use xml.

The format of xml is as follows, which distinguishes data structures by <> nodes:

<?xml version="1.0"?>
    <country name="Liechtenstein">
        <rank updated="yes">2</rank>
        <neighbor name="Austria" direction="E"/>
        <neighbor name="Switzerland" direction="W"/>
    <country name="Singapore">
        <rank updated="yes">5</rank>
        <neighbor name="Malaysia" direction="N"/>
    <country name="Panama">
        <rank updated="yes">69</rank>
        <neighbor name="Costa Rica" direction="W"/>
        <neighbor name="Colombia" direction="E"/>

The XML protocol is supported in all languages. In python, the following modules can be used to manipulate xml.

import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml")
root = tree.getroot()
#Traversing xml documents
for child in root:
    print(child.tag, child.attrib)
    for i in child:
#Traverse year nodes only
for node in root.iter('year'):

Modify and delete xml document content

import xml.etree.ElementTree as ET
tree = ET.parse("xmltest.xml")
root = tree.getroot()
for node in root.iter('year'):
    new_year = int(node.text) + 1
    node.text = str(new_year)
#Delete node
for country in root.findall('country'):
   rank = int(country.find('rank').text)
   if rank > 50:

Create your own xml document

import xml.etree.ElementTree as ET
new_xml = ET.Element("namelist")
name = ET.SubElement(new_xml,"name",attrib={"enrolled":"yes"})
age = ET.SubElement(name,"age",attrib={"checked":"no"})
sex = ET.SubElement(name,"sex")
sex.text = '33'
name2 = ET.SubElement(new_xml,"name",attrib={"enrolled":"no"})
age = ET.SubElement(name2,"age")
age.text = '19'
et = ET.ElementTree(new_xml) #Generating Document Objects
et.write("test.xml", encoding="utf-8",xml_declaration=True)
ET.dump(new_xml) #Print-generated format

9. PyYAML module

Python can also easily handle ymal document formats, except that you need to install a module to refer to documents:

10. ConfigParser module

For generating and modifying common configuration documents, the name of the current module is changed to configparser in Python version 3.x.

Look at the common document formats for a lot of software as follows

ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
User = hg
Port = 50022
ForwardX11 = no

What if you want to generate such a document with python?

import configparser
config = configparser.ConfigParser()
config["DEFAULT"] = {'ServerAliveInterval': '45',
                      'Compression': 'yes',
                     'CompressionLevel': '9'}
config[''] = {}
config['']['User'] = 'hg'
config[''] = {}
topsecret = config['']
topsecret['Host Port'] = '50022'     # mutates the parser
topsecret['ForwardX11'] = 'no'  # same here
config['DEFAULT']['ForwardX11'] = 'yes'
with open('example.ini', 'w') as configfile:

You can read it again after you finish it.

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.sections()
>>> config.sections()
['', '']
>>> '' in config
>>> '' in config
>>> config['']['User']
>>> config['DEFAULT']['Compression']
>>> topsecret = config['']
>>> topsecret['ForwardX11']
>>> topsecret['Port']
>>> for key in config['']: print(key)
>>> config['']['ForwardX11']

Configgparser Addendum, Delete and Change Check Syntax

k1 = v1
k1 = v1
import ConfigParser
config = ConfigParser.ConfigParser()'i.cfg')
# ########## read ##########
#secs = config.sections()
#print secs
#options = config.options('group2')
#print options
#item_list = config.items('group2')
#print item_list
#val = config.get('group1','key')
#val = config.getint('group1','key')
# ########## rewrite ##########
#sec = config.remove_section('group1')
#config.write(open('i.cfg', "w"))
#sec = config.has_section('wupeiqi')
#sec = config.add_section('wupeiqi')
#config.write(open('i.cfg', "w"))
#config.write(open('i.cfg', "w"))
#config.write(open('i.cfg', "w"))

11. hashlib module

For encryption-related operations, 3.x replaces MD5 module and sha module, mainly providing SHA1, SHA224, SHA256, SHA384, SHA512, MD5 algorithm.

import hashlib
m = hashlib.md5()
m.update(b"It's me")
m.update(b"It's been a long time since last time we ...")
print(m.digest()) #Binary format hash
print(len(m.hexdigest())) #Hexadecimal format hash
def digest(self, *args, **kwargs): # real signature unknown
    """ Return the digest value as a string of binary data. """
def hexdigest(self, *args, **kwargs): # real signature unknown
    """ Return the digest value as a string of hexadecimal digits. """
import hashlib
# ######## md5 ########
hash = hashlib.md5()
# ######## sha1 ########
hash = hashlib.sha1()
# ######## sha256 ########
hash = hashlib.sha256()
# ######## sha384 ########
hash = hashlib.sha384()
# ######## sha512 ########
hash = hashlib.sha512()

Not enough hanging? python also has an hmac module that internally processes the key s and content we create and then encrypts them

Hash message authentication code (HMAC) is an authentication mechanism based on message authentication code MAC (Message Authentication Code). When using HMAC, both sides of message communication can authenticate the authenticity of the message by verifying the authentication key K added in the message.

Generally used for message encryption in network communication, the premise is that the two sides should first agree on the key, just like the connection password, and then the message is encrypted by key in message sending, and the receiver encrypts the message with key + message plaintext, then the encrypted value is equal to the relative ratio of the sender, so as to verify the authenticity of the message and send it. The legitimacy of the person.

import hmac
h ='King of Heaven and Tiger of the Earth', b'Pagoda will stop river monster')
print h.hexdigest()

For more articles on md5,sha1,sha256, etc., see here https://www.tbs-certificates....

12. re module

Common regular expression symbols

'.'By default, any character except n is matched, and if flag DOTALL is specified, any character, including newline, is matched.
'^'matches the beginning of the character. If flags MULTILINE is specified, this can also be matched on (r"^a", nabc neee",  flags = re. MULTILINE)
'$'matches the end of the character, or e. search ("foo$", "bfoo nsdfsf", "flags = re. MULTILINE). group () can also be used.
'*'matches the character before the * sign 0 or more times, and re.findall("ab*","cabb3abcbbac") results in ['abb','ab','a']
'+'matches the previous character one or more times, and re.findall("ab+","ab+cd+abb+bba") results ['ab','abb']
'?'Matches the previous character 1 or 0 times
 '{m}'matches the previous character m times
 '{n,m}'matches the previous character n to m times, and re.findall("ab{1,3}","abb abc abbcbbb") results in'abb','ab','abb','abb']
'|'matches | left or | right characters,"abc|ABC","ABCBabcCD").group() results in'ABC'
'(...)'grouping matching, re. search ("(abc) {2} a (123 | 456) c", "abcabca 456c"). group () results abcabca 456c
' A'matches only from the beginning of the character, but re. search (" Aabc", "alexabc") does not match.
' Z'matches the end of the character.$
'd'matching number 0-9
 'D'matching non-numeric
 ' w'matching [A-Za-z0-9]
'W'Matched Non[A-Za-z0-9]
's'matches blank characters, \ t,  n,  r, re. search (" s+", "ab tc1 n3"). group () result' t'
Group matching ("(? P < province > [0-9] {4}) (? P < City > [0-9] {2}) (? P < birthday > [0-9] {4})", "371481199306143242"). group Dict ("city") results {'province':'3714','city':'81','birthday':'1993'}

The most commonly used matching grammar

re.match matches from scratch matching contains
 re.findall puts all matched characters back in the elements in the list
 re.splitall uses matched characters as list separators
 re.sub Matches Characters and Replaces

The Puzzlement of Backslash
Like most programming languages, regular expressions use "" as an escape character, which can cause backslash problems. If you need to match the character "" in the text, four backslashes \"will be required in regular expressions expressed in programming languages: the first two and the last two are used to escape into backslashes in programming languages, convert into two backslashes, and then convert into a backslash in regular expressions. Native strings in Python solve this problem very well. The regular expressions in this example can be represented by r "". Similarly, the "d" matching a number can be written as r"d". With native strings, you don't have to worry about missing the backslash, and the expression is more intuitive.

A couple of matching patterns that you just need to know lightly

re.I(re.IGNORECASE): Ignore case and case (complete in parentheses, the same below)
M(MULTILINE): Multi-line mode, changing the behavior of'^'and'$' (see figure above)
S(DOTALL): Change the behavior of'. 'by arbitrary matching patterns of points

Tags: Python xml shell SHA1

Posted on Sun, 08 Sep 2019 03:14:00 -0700 by anand