Web Crawler Learning Notes

Web Crawler Learning Notes (2)

1 Data

  • "Python Web Crawler from Introduction to Practice" Tang Song, Chen Zhiquan. Mainly for Python 3 under windows platform.
  • Lessons on Rookie Birds - "python"

2 Notes

2-1 Python Foundation (1)++.

  1. python's list also has a read-only format, Tuple, which assigns values with () and only at initialization.
  2. The list has the built-in method append ()
  3. The dictionary has built-in methods keys() (get the values of all keys), values()
  4. Tuples are marked by "()", lists by "[]" and dictionaries by "{}".
  5. Type (variable) looks at the data type of the variable and can also use isinstance (variable, type) to determine which type it is. type() does not consider a subclass to be a parent type, isinstance() does consider a subclass to be a parent type.

2-2 Python Foundation (2)

  1. Conditional statement - Format is if condition: instead of () in Java, one is final: there is also an else (but there is a:) after the else, and there is also an elif abbreviated from else if.
a = 'book'
b = 'java'
c = "book"
if a == b :
else :
if a == c :
elif b == c :
else :
  1. Loop statement - There are two kinds of loops commonly used, for loops and while loops, which are also () turned to the end:. However, python's for loop is closer to the foreach loop in form, and the loop in Python can also use else.
myBook = ['java', 'python', 'c']
for book in myBook:
for i,j in enumerate(('a','b','c')):
    print (i,j)
for i,j in enumerate(myBook):
    print (i,j)
count = 0
while count < 3:
    count = count + 1

The for loop uses built-in functions len() and range(), which return the length of the list, that is, the number of elements, and range, which returns the number of a sequence. (Examples from rookies)

fruits = ['banana', 'apple',  'mango']
for index in range(len(fruits)):
   print 'Current fruits :', fruits[index]

else uses examples (examples from novice birds)

for num in range(10,20):  # Iteration numbers between 10 and 20
   for i in range(2,num): # Iteration by factor
      if num%i == 0:      # Determine the first factor
         j=num/i          # Calculate the second factor
         print '%d Be equal to %d * %d' % (num,i,j)
         break            # Jump out of the current loop
   else:                  # else part of the loop
      print num, 'It's a prime number.'

In addition, you can use continue, break, pass (do nothing) in the loop.
3. Functions - the contents of parameters (() must be correctly written into functions

def add(x):
    y = x + 1
    return y
w = 1
  1. Operator, let's just mention that in python there are ** - power - returning y power of x, //- dividing - returning quotient integer part (downward integer) and no ++, - and -. In addition, and, or, not are written separately. python also has member operators in, not in; identity operators is, not is (storage unit for comparing two objects)
    1. class
class Person:
    #This is a class variable whose value will be shared among all instances of this class
    n = 0 
    #_ The init_ construction method, notice that there are two before and after._
    #Self represents an instance of a class, and self is necessary to define the method of a class.
    #Although it is not necessary to pass in the appropriate parameters when invoking.
    def __init__(self, name, age): 
        self.name = name
        self.age = age
    def detail(self): #Call encapsulated content through this function
obj1 = Person('sss', 16)#Object declaration

Inheritance Format

Class subclass name (parent class name):

Use super keyword to use parent class construction method when overwriting construction method

class Father(object):
    def __init__(self, name):
class Son(Father):
    def __init__(self, name):
    super(Son, self).__init__(name)
    1. Foundation overloading method
      The following table lists some common functions that can be overridden in their own classes:
__init__ ( self [,args...] )
 Simple invocation method: obj = className(args)
__del__( self )
Destruction, deleting an object
 Simple invocation method: del obj
__repr__( self )
Converted to form for interpreter reading
 Simple invocation method: repr(obj)
__str__( self )
Used to convert values into human-readable forms
 Simple invocation method: str(obj)
__cmp__ ( self, x )
Object comparison
 Simple invocation method: cmp(obj, x)
  1. Module - A Python file that ends with. py and contains Python object definitions and Python statements. Use import statements to introduce modules.
# Import module
import support 
# Now you can call the functions contained in the module

Tags: Python Java Windows

Posted on Wed, 04 Sep 2019 22:53:00 -0700 by JayFM