Recently, I have just completed a project about big data, and the framework used in this project includes spring boot. Because it is an offline data analysis, Hive is also selected for component selection (Spark or HBase may be used for real-time ) This blog is about how to configure Hive in the spring ...
Posted on Wed, 12 Feb 2020 10:19:54 -0800 by mrmom
1. Built in functions of Hive system
1.1 numerical calculation function
1. Rounding function: round
Syntax: round(double a)Return value: BIGINTNote: returns the integer value part of double type (following rounding)
hive> select round(3.1415926) from tableName;
hive> select round(3.5) from tableName;
hive> create table tableName a ...
Posted on Wed, 05 Feb 2020 04:14:30 -0800 by AndrewBacca
Data Analysis and Forecast of Taobao Shuang11
The system and software involved in this case:
Linux System (CENTOS 7)
Posted on Thu, 23 Jan 2020 01:15:49 -0800 by designedfree4u
VX: Data Science Lecture
1. Prepare hive installation package
Download the hive installation package according to the 1.1 tutorial
1.1 download hive
Download address After opening the download address, click apache-hive-1.2.2-bin.tar.gz as shown below to download
1.2 upload hvie installation package
Based on our previous environment installati ...
Posted on Wed, 22 Jan 2020 09:20:36 -0800 by $var
1 purpose of document preparation
In the previous article, Fayson introduced< 0491 - how to install CDH6.1 in RedHat 7.4 >, here we start to install Kerberos based on this environment. Kerberos is a third-party protocol for security authentication, which is not dedicated to Hadoop. You can also ...
Posted on Sat, 11 Jan 2020 22:18:12 -0800 by tonchily
stay Last article In this article, we describe the Query Plan and Execution Summary sections of Profile.
Profile's query plan and execution summary is as follows:
Max Per-Host Resource Reservation: Memory=0B
Per-Host Resource Estimates: ...
Posted on Tue, 10 Dec 2019 04:42:12 -0800 by Calamity-Clare
Many Impala users don't know how to read the Impala query profile to see what's going on behind a query, and then tune the query to get the most out of it.So I want to write a simple article to share my experience and hope that it will help people who want to know more.
This is the first part of this series, and I'll introduce some of the basic ...
Posted on Sat, 07 Dec 2019 22:29:06 -0800 by RW
drwxrwxr-x 8 hadoop hadoop 4096 Apr 16 04:45 ./
drwxr-xr-x 28 hadoop hadoop 4096 Apr 16 07:04 ../
drwxrwxr-x 3 hadoop hadoop 4096 Apr 16 04:45 bin/
drwxrwxr-x 2 hadoop hadoop 4096 Apr 16 07:02 conf/
drwxrwxr-x 4 hadoop hadoop 4096 Apr 16 04:45 examples/
Posted on Wed, 27 Nov 2019 12:35:35 -0800 by dr4296
oozie create workflow
The execution command of the workflow refers to the blog: https://www.jianshu.com/p/6cb3a4b78556 , or type oozie help for help
Manually configure the workflow of oozie
The job.properties file holds some parameters that may be used in the workflow.xml filejob.properties
# Note that the variable name should not contain speci ...
Posted on Sun, 10 Nov 2019 10:47:21 -0800 by FlashHeart
After starting spark shell, query the table information in hive and report an error
spark.sql("select * from student.student ").show()
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStor ...
Posted on Wed, 06 Nov 2019 08:12:21 -0800 by Jagarm