Data visualization codeplaces rating

Display page Code repository

I met at station B recently [data visualization] Codeforces history top 10 I thought it was very interesting, so I had the idea to create a visual rating for our school team.

UP main project is based on a hot project on GitHub Historical-ranking-data-visualization-based-on-d3.js It can convert historical data ranking into dynamic histogram chart, which is based on D3.

D3js: data driven documents is a JavaScript library that can operate documents based on data. It can help you use HTML, CSS, SVG and Canvas to present data. D3 follows the existing Web standards and can run independently in modern browsers without any other framework. It combines powerful visualization components to drive DOM operations.

D3, as a powerful data visualization function library, has derived many excellent and practical wheels. The above dynamic histogram chart is a model.

get down to business. First of all, we need to get the history of the school team members' participation in codeplaces, mainly including the game number, time, and changes before and after rating. We can get this information through Python crawler, but codeplaces has provided a convenient interface (refer to API document )Beautiful and Zizi.

Before that, we have prepared two lists, hands. JSON and names.json.

// handles.json
[
    "handle1",
    "handle2",
    "handle3",
    "// ..."
]
// names.json
{
    "handle1": "name1",
    "handle2": "name2",
    "handle3": "name3",
    "//": "..."
}

Then write the initial data file data.josn from the API filtering information through the Python requests method.

# getData.py
import requests
import json
import time
import datetime

STARTTIME = datetime.datetime(2019, 5, 1)


def getDay(x):
    xTime = time.strftime("%Y %m %d", time.localtime(x)).split()
    xTime = datetime.datetime(int(xTime[0]), int(xTime[1]), int(xTime[2]))
    return (xTime - STARTTIME).days

NOW = getDay(time.time())

with open("handles.json", "r") as handlesjson:
    handles = json.load(handlesjson)

data = {}

for i in handles:
    print(i)
    while True:
        try:
            response = requests.get(
                "https://codeforces.com/api/user.rating?handle=" + i)
            userdata = json.loads(response.text)["result"]
        except:
            time.sleep(1)
        else:
            break
    old = 0
    data.setdefault(i, [])
    for j in userdata:
        new = getDay(j["ratingUpdateTimeSeconds"])
        if new >= 0:
            rating = j["oldRating"]
            for k in range(old, new):
                data[i].append(rating)
            old = new
    rating = userdata[-1]["newRating"]
    for k in range(old, NOW + 1):
        data[i].append(rating)

with open("data.json", "w") as datajson:
    datajson.write(json.dumps(data))

Then the name mapping in names.json is integrated to convert data.json into the final data file data.csv for D3 parsing.

# json2csv.py
import json
import time
import operator
import datetime

MAXPERDAY = 10

with open("handles.json", "r") as handlesjson:
    handles = json.load(handlesjson)
with open("data.json") as datajson:
    data = json.load(datajson)
with open("names.json") as namesjson:
    names = json.load(namesjson)

STARTTIME = datetime.datetime(2019, 5, 1)

f = open("data.csv", "w")
f.write("name,type,value,date\n")

old = []

for i in range(len(data["156250lcy"])):
    temp = []
    tim = (STARTTIME + datetime.timedelta(days=i)).strftime("%Y/%m/%d")
    for j in handles:
        temp.append({"handle": j, "rating": data[j][i]})
    temp.sort(key=lambda x: x["rating"], reverse=True)
    if operator.eq(temp[:MAXPERDAY], old[:MAXPERDAY]):
        continue
    for j in range(MAXPERDAY):
        u = temp[j]
        f.write(u["handle"] + "," + names[u["handle"]] + "," +
                str(u["rating"]) + "," + tim + "\n")
    old = temp

So far, all valid data has been obtained, and the dynamic chart is finally configured. Change the Web page and adjust parameters, omitted. be accomplished.

Released 408 original articles, praised 149, visited 240000+
Private letter follow

Tags: JSON Python github Javascript

Posted on Mon, 16 Mar 2020 01:15:39 -0700 by daniel1988