Experience in 2020 Spring Festival python crawler dictionary

First of all, I want to express my gratitude to the heroes who fought in the front line.
We will get through this epidemic soon

My Spring Festival is really boring, so I'm going to open a little turtle's python class.
Then I saw that many people said no, or the delete method is to remove the o in the request url and ignore its anti crawler mechanism
Then I compare the iron head. My head is really iron. My height of 1.7 meters must compete with that of 1.88 meters

Yesterday Arsenal were 4-0 and I was happy.
Saka is tough, zaca is tough. Pepe finally played his 80 million worth and loved him.
Aubamei calfskin
Then I love lacazette. He can score all the balls. It's really awesome!
I'll die here again
This is my first post, I am very locomotive, a lot of nonsense.

I really don't want to be aggrieved, so this tutorial appears
No bullshit, dry goods

First of all, I found a blog in 2018:
https://blog.csdn.net/li939891142/article/details/84592442
 First of all, thank him here
 But now his code is dead

According to the predecessor's methodology, we can't work now. We should find a proper js encryption method, so we find
http://fanyi.youdao.com/ 
Press F12 to find the elem block and js
 Pictured


After clicking in, we found that it was a pile of code, which could not be seen at all. We need tools to deal with js code. We need Baidu to format js code
Then save it in a word document and search for salt
As shown in the picture:

We need to start comparing:
First of all, the sign is very different. The string of the fourth parameter has changed obviously.
And then the way salt was generated changed
More bv and ts
After knowing this, we can forge these parameters and cheat the server that we are normal people
Code will be attached at the end

Let me mention this bv here
var t = n.md5(navigator.appVersion),
This t is bv.

By the way, this correspondence is from
		return {
	            ts: r,
	            bv: t,
	            salt: i,
	            sign: n.md5("fanyideskweb" + e + i + "n%A-rKaT5fb[Gy?;N5@Tj")
	        }
As you can see here.

I don't know how to return python to the appversion of the server, so I use js (press f12) directly
 Then as shown in the figure:

My code gives this string of characters directly to copy
 Mo Xiao
 Next, I changed the ugly code
"""
//Through searching, you can find the operation code in js code
1.This is calculation. salt In the fanyi.min.js Found in file,
t = "" + ((new Date).getTime() + parseInt(10 * Math.random(), 10));
https://blog.csdn.net/li939891142/article/details/84592442
2.sign: n.md5("fanyideskweb" + e + t + "sr_3(QOHT)L2dx#uuGR@r")
md5 There are four parameters needed. The first and the fourth are fixed strings. The third is called salt. The second is the input of the word to find
"""

def getSalt():
    """
    //This is the formula for calculating salt "" + ((new Date).getTime() + parseInt(10 * Math.random(), 10));
    //Translate it into Python code
    :return:
    """
    import time,random
    # The time stamp generated by (new Date).getTime() is inconsistent with the unit of time.time(), so it needs to be multiplied by 1000
    r= '' + str(str(int(time.time()*1000)))
    salt = r + str(random.randint(0,10))

    return salt

def getMd5(v):
    import hashlib
    md5 = hashlib.md5()

    # A parameter in bytes format is required
    md5.update(v.encode("utf-8"))
    sign = md5.hexdigest()

    return sign



def getSign(key, salt):
    sign = "fanyideskweb" + key + str(salt) + "n%A-rKaT5fb[Gy?;N5@Tj"
    sign = getMd5(sign)
    return sign

def getbv():
    appVersion='5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.106 Safari/537.36'
    bv = getMd5(appVersion)
    return bv

def getts():
    import time, random
    # The time stamp generated by (new Date).getTime() is inconsistent with the unit of time.time(), so it needs to be multiplied by 1000
    r = '' + str(str(int(time.time() * 1000)))
    return r


from urllib import request,parse

def youdao(key):

    url = "http://fanyi.youdao.com/translate_o?smartresult=dict&smartresult=rule"
    salt = getSalt()
    data = {
        "i": key,
        "from": "AUTO",
        "to": "AUTO",
        "smartresult": "dict",
        "client": "fanyideskweb",
        "salt": str(salt),
        "sign": getSign(key, salt),
        'ts' : getts(),
        'bv' : getbv(),
        "doctype": "json",
        "version": "2.1",
        "keyfrom": "fanyi.web",
        "action": "FY_BY_CLICKBUTTION",
    }
    print(data)
    # Parameter data needs to be in bytes format
    data = parse.urlencode(data).encode('utf-8')
    headers = {
        "Accept": "application/json,text/javascript,*/*;q=0.01",
        # "Accept-Encoding": "gzip,deflate",
        "Accept-Language": "zh-CN,zh;q=0.9",
        "Connection": "keep-alive",
        # "Content-Length": len(data),
        "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
        "Cookie": "OUTFOX_SEARCH_USER_ID=1076696971@10.168.8.63;JSESSIONID=aaaZ6s5m9DVmYf8g3QoDw;OUTFOX_SEARCH_USER_ID_NCOO=473292646.5080033;___rl__test__cookies=1543228484656",
        "Host": "fanyi.youdao.com",
        "Origin":"http://fanyi.youdao.com",
        "Referer":"http://fanyi.youdao.com/",
        "User-Agent": "Mozilla/5.0(WindowsNT10.0;Win64;x64)AppleWebKit/537.36(KHTML,likeGecko)Chrome/69.0.3497.100Safari/537.36OPR/56.0.3051.104X-Requested-With:XMLHttpRequest"
    }

    req = request.Request(url=url, data=data, headers=headers)
    rsp = request.urlopen(req)
    html = rsp.read().decode('utf-8')
    # for values in html["translateResult"]:
    #     print(values)

    print(html)

if __name__ == '__main__':
    youdao("girl")
	#     Reference: https://blog.csdn.net/li939891142/article/details/84592442 blogger thanks him

I'd like to conclude with the following results:

In addition, I want to drink a bottle of coke. You are kind enough to give me a few cents. Maybe I can get a bottle of coke.

Published 1 original article, praised 0 and visited 5
Private letter follow

Tags: Python JSON Spring Windows

Posted on Mon, 17 Feb 2020 02:50:24 -0800 by sphinx9999