Python 3 uses selenium + Chrome basic operation code

selenium is a third-party library of Python, which needs to be installed before use. But if you use anaconda, you can omit this step. Why? Bring your own, willful.

Installation command:

pip install selenium

(1) Use selenium to open the specified website. Take Taobao for example.

# -*- coding: utf-8 -*-
"""
Created on Wed Jul 25 10:12:39 2018
@author: brave_man
email: 1979887709@qq.com
"""

from selenium import webdriver
from time import sleep

b = webdriver.Chrome()
b.get("http://www.taobao.com")

elem = b.find_element_by_id('q')
elem.send_keys('iphone')
sleep(3)
elem.clear()
elem.send_keys("ipad")
button = b.find_element_by_class_name("btn-search")
button.click()
sleep(5)
b.close()

(2) Simple drag action (for verification code identification)

# -*- coding: utf-8 -*-
"""
Created on Wed Jul 25 15:00:10 2018
@author: brave_man
email: 1979887709@qq.com
"""

from selenium import webdriver
from selenium.webdriver import ActionChains

b = webdriver.Chrome()
url = "http://www.runoob.com/try/try.php?filename=jqueryui-api-droppable"
b.get(url)
b.switch_to.frame('iframeResult')
sou = b.find_element_by_css_selector('#draggable')
tar = b.find_element_by_css_selector('#droppable')
actions = ActionChains(b)
actions.drag_and_drop(sou, tar)
actions.perform()

(3) In crawlers, it may be affected by external factors such as network speed, which may lead to failure in obtaining web page elements. Here are two waiting modes

1. Implicit wait: webdriver does not find the desired element in DOM. After waiting for the specified time, it will throw an exception that the specified element cannot be found. It can be used when the network speed is very slow

from selenium import webdriver

b = webdriver.Chrome()
b.implicitly_wait(10)
b.get("https://www.zhihu.com/explore")
elem = b.find_element_by_class_name('zu-top-add-question')
print(elem)

2. Explicit waiting

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

b = webdriver.Chrome()
#b.implicitly_wait(10)
b.get("https://taobao.com/")
#elem = b.find_element_by_class_name('zu-top-add-question')
b_wait = WebDriverWait(b, 10)
elem = b_wait.until(EC.presence_of_all_elements_located((By.ID, 'q')))
button = b_wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.btn-search')))
print(elem, button)

(4) Forward and backward

from selenium import webdriver
from time import sleep

b = webdriver.Chrome()
b.get("http://www.baidu.com")
sleep(1)
b.get("http://www.sina.com.cn")
sleep(1)
b.back()
sleep(3)
b.forward()
sleep(3)
b.close()

For more information, please refer to the document: http://selenium-python-zh.readthedocs.io/en/latest/index.html

Tags: Python Selenium network Anaconda

Posted on Fri, 31 Jan 2020 15:20:15 -0800 by Lerris