Using python to record problems encountered in budget system

1, Use df.loc [] to query by criteria

  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 2434, in main
    rev = ExpenseApiDown(cfg_path, log=logging)
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 793, in __init__
    self.done = self.split()  # Level judgment is needed in merging to generate level.json
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 1288, in split
    done = self.boss_split()
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 1617, in boss_split
    self.get_two_rounds_for_data()
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 1234, in get_two_rounds_for_data
    df = self.calculate_two_rounds_of_data(input_sheet_name,columns_dict)
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 1201, in calculate_two_rounds_of_data
    df1 = self.two_rounds_util_for_president_df(df1,df2,sale_rate_df,screen_column,screen_column_value,columns_dict)
  File "/Users/jiangmengliang/PycharmProjects/jiang.ml/budget/cost_budget/expense_down_api.py", line 1058, in two_rounds_util_for_president_df
    df1.loc[(df1["brand_no"] == brand_no) & (df1["fy_year"] == self.fin_year), column] = result
  File "/Users/jiangmengliang/PycharmProjects/workspacespython3/venv/lib/python3.7/site-packages/pandas/core/ops/__init__.py", line 1229, in wrapper
    res = na_op(values, other)
  File "/Users/jiangmengliang/PycharmProjects/workspacespython3/venv/lib/python3.7/site-packages/pandas/core/ops/__init__.py", line 1115, in na_op
    result = method(y)
FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison

FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison

The reason for this warning:

1. Multiple conditions are connected with & but conditions are not enclosed with ().

Example: DF1. LOC [(DF1 ["brand no"] = = brand no) & DF1 ["FY year"] = = self. Fin_year, column]

2. In df, fy'year is converted to number type by default, but self.fin'year is a string type. The type is not unified for comparison, and python will automatically recognize it, but this uncertainty will make Python give a warning.

2, pd.merge(df1,df2,how='left',on=[columns]) is used to match excel across sheet s

3, df.groupby([columns]).sum().reset_index() is used to simply summarize these fields according to columns. sum() will automatically group and sum the data types

4, json and df convert to each other

json.loads(df.to_json(orient="records"), encoding="utf8")#Convert df to json string

Where the specified original = "records" can generate key value pairs by column name and value

df1 = pd.read_json(json.dumps(data_json, ensure_ascii=False), orient="records")

This is the key read by json as df, where the column name is: key value pair

Where ensure = false is used to specify that the code does not use the default ASCII code

5, Append one df to the other

result_df = df1.append(df2, ignore_index=True, sort=True)

ignore_index = True is appended according to the same column name

sort = True means that the row number is added to the back row

Tags: Big Data JSON Python Excel encoding

Posted on Tue, 14 Apr 2020 08:06:18 -0700 by grahamb314