标签：Hadoop

Impala：查询Hive中设定的表的数据

Adamhuan 2018年8月31日 No Comments

命令行如下：

[root@cdh2 ~]# impala-shell
Starting Impala Shell without Kerberos authentication
Connected to cdh2:21000
Server version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2)
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v2.7.0-cdh5.10.0 (785a073) built on Fri Jan 20 12:03:56 PST 2017)

Want to know what version of Impala you're connected to? Run the VERSION command to
find out!
***********************************************************************************
[cdh2:21000] > connect cdh2;
Connected to cdh2:21000
Server version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2)
[cdh2:21000] > 
[cdh2:21000] > invalidate metadata;
Query: invalidate metadata
Query submitted at: 2018-08-31 23:28:25 (Coordinator: http://cdh2:25000)
Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=724a13ec400f3864:ceb32a4400000000

Fetched 0 row(s) in 6.92s
[cdh2:21000] > 
[cdh2:21000] > show tables;
Query: show tables
+-------------+
| name        |
+-------------+
| movie       |
| movierating |
+-------------+
Fetched 2 row(s) in 0.54s
[cdh2:21000] > 
[cdh2:21000] > select count(*) from movie;
Query: select count(*) from movie
Query submitted at: 2018-08-31 23:29:33 (Coordinator: http://cdh2:25000)
Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=824a3ba5793a8adf:4cd33f1700000000
+----------+
| count(*) |
+----------+
| 3881     |
+----------+
Fetched 1 row(s) in 18.26s
[cdh2:21000] > 
[cdh2:21000] > select count(*) from movierating;
Query: select count(*) from movierating
Query submitted at: 2018-08-31 23:30:03 (Coordinator: http://cdh2:25000)
Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=304df122f17f2a8a:8592d4b00000000
+----------+
| count(*) |
+----------+
| 1000205  |
+----------+
Fetched 1 row(s) in 4.25s
[cdh2:21000] > 
[cdh2:21000] > quit;
Goodbye root
[root@cdh2 ~]#

[root@cdh2 ~]# impala-shell

Starting Impala Shell without Kerberos authentication

Connected to cdh2:21000

Server version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2)

***********************************************************************************

Welcome to the Impala shell.

(Impala Shell v2.7.0-cdh5.10.0 (785a073) built on Fri Jan 20 12:03:56 PST 2017)

Want to know what version of Impala you're connected to? Run the VERSION command to

find out!

***********************************************************************************

[cdh2:21000] > connect cdh2;

Connected to cdh2:21000

Server version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2)

[cdh2:21000] >

[cdh2:21000] > invalidate metadata;

Query: invalidate metadata

Query submitted at: 2018-08-31 23:28:25 (Coordinator: http://cdh2:25000)

Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=724a13ec400f3864:ceb32a4400000000

Fetched 0 row(s) in 6.92s

[cdh2:21000] >

[cdh2:21000] > show tables;

Query: show tables

+-------------+

| name |

+-------------+

| movie |

| movierating |

+-------------+

Fetched 2 row(s) in 0.54s

[cdh2:21000] >

[cdh2:21000] > select count(*) from movie;

Query: select count(*) from movie

Query submitted at: 2018-08-31 23:29:33 (Coordinator: http://cdh2:25000)

Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=824a3ba5793a8adf:4cd33f1700000000

+----------+

| count(*) |

+----------+

| 3881 |

+----------+

Fetched 1 row(s) in 18.26s

[cdh2:21000] >

[cdh2:21000] > select count(*) from movierating;

Query: select count(*) from movierating

Query submitted at: 2018-08-31 23:30:03 (Coordinator: http://cdh2:25000)

Query progress can be monitored at: http://cdh2:25000/query_plan?query_id=304df122f17f2a8a:8592d4b00000000

+----------+

| count(*) |

+----------+

| 1000205 |

+----------+

Fetched 1 row(s) in 4.25s

[cdh2:21000] >

[cdh2:21000] > quit;

Goodbye root

[root@cdh2 ~]#

CM的图形界面中，可以看到上述命令的历史：

——————————————————
Done。

Cloudera CDH, Cloudera, Hadoop, IMPALA

Hive Beeline：表数据为NULL

Adamhuan 2018年8月31日 No Comments

在使用HIVE的Beeline访问数据表的时候，有可能出现如题所示的NULL的问题。

具体如下：

[hdfs@cdh1 ~]$ beeline -u jdbc:hive2://cdh1:10000 -n hdfs
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
scan complete in 3ms
Connecting to jdbc:hive2://cdh1:10000
Connected to: Apache Hive (version 1.1.0-cdh5.10.0)
Driver: Hive JDBC (version 1.1.0-cdh5.10.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.10.0 by Apache Hive
0: jdbc:hive2://cdh1:10000> 
0: jdbc:hive2://cdh1:10000> show tables;
INFO  : Compiling command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e): show tables
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e); Time taken: 0.009 seconds
INFO  : Executing command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e): show tables
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e); Time taken: 0.202 seconds
INFO  : OK
+--------------+--+
|   tab_name   |
+--------------+--+
| movie        |
| movierating  |
+--------------+--+
2 rows selected (0.375 seconds)
0: jdbc:hive2://cdh1:10000> 
0: jdbc:hive2://cdh1:10000> select * from movie limit 5;
INFO  : Compiling command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c): select * from movie limit 5
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:movie.id, type:int, comment:null), FieldSchema(name:movie.name, type:string, comment:null), FieldSchema(name:movie.year, type:smallint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c); Time taken: 0.06 seconds
INFO  : Executing command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c): select * from movie limit 5
INFO  : Completed executing command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c); Time taken: 0.017 seconds
INFO  : OK
+-----------+-------------+-------------+--+
| movie.id  | movie.name  | movie.year  |
+-----------+-------------+-------------+--+
| NULL      | NULL        | NULL        |
| NULL      | NULL        | NULL        |
| NULL      | NULL        | NULL        |
| NULL      | NULL        | NULL        |
| NULL      | NULL        | NULL        |
+-----------+-------------+-------------+--+
5 rows selected (0.167 seconds)
0: jdbc:hive2://cdh1:10000> 
0: jdbc:hive2://cdh1:10000> select * from movierating limit 5;
INFO  : Compiling command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d): select * from movierating limit 5
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:movierating.userid, type:int, comment:null), FieldSchema(name:movierating.movieid, type:string, comment:null), FieldSchema(name:movierating.rating, type:tinyint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d); Time taken: 0.058 seconds
INFO  : Executing command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d): select * from movierating limit 5
INFO  : Completed executing command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d); Time taken: 0.01 seconds
INFO  : OK
+---------------------+----------------------+---------------------+--+
| movierating.userid  | movierating.movieid  | movierating.rating  |
+---------------------+----------------------+---------------------+--+
| NULL                | NULL                 | NULL                |
| NULL                | NULL                 | NULL                |
| NULL                | NULL                 | NULL                |
| NULL                | NULL                 | NULL                |
| NULL                | NULL                 | NULL                |
+---------------------+----------------------+---------------------+--+
5 rows selected (0.17 seconds)
0: jdbc:hive2://cdh1:10000>

[hdfs@cdh1 ~]$ beeline -u jdbc:hive2://cdh1:10000 -n hdfs

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0

scan complete in 3ms

Connecting to jdbc:hive2://cdh1:10000

Connected to: Apache Hive (version 1.1.0-cdh5.10.0)

Driver: Hive JDBC (version 1.1.0-cdh5.10.0)

Transaction isolation: TRANSACTION_REPEATABLE_READ

Beeline version 1.1.0-cdh5.10.0 by Apache Hive

0: jdbc:hive2://cdh1:10000>

0: jdbc:hive2://cdh1:10000> show tables;

INFO : Compiling command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e): show tables

INFO : Semantic Analysis Completed

INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)

INFO : Completed compiling command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e); Time taken: 0.009 seconds

INFO : Executing command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e): show tables

INFO : Starting task [Stage-0:DDL] in serial mode

INFO : Completed executing command(queryId=hive_20180831151616_72e0d1c3-14f0-4410-879c-c50adf08ea5e); Time taken: 0.202 seconds

INFO : OK

+--------------+--+

| tab_name |

+--------------+--+

| movie |

| movierating |

+--------------+--+

2 rows selected (0.375 seconds)

0: jdbc:hive2://cdh1:10000>

0: jdbc:hive2://cdh1:10000> select * from movie limit 5;

INFO : Compiling command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c): select * from movie limit 5

INFO : Semantic Analysis Completed

INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:movie.id, type:int, comment:null), FieldSchema(name:movie.name, type:string, comment:null), FieldSchema(name:movie.year, type:smallint, comment:null)], properties:null)

INFO : Completed compiling command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c); Time taken: 0.06 seconds

INFO : Executing command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c): select * from movie limit 5

INFO : Completed executing command(queryId=hive_20180831151717_b0ee2167-b317-4812-9f34-0e868d45463c); Time taken: 0.017 seconds

INFO : OK

+-----------+-------------+-------------+--+

| movie.id | movie.name | movie.year |

+-----------+-------------+-------------+--+

| NULL | NULL | NULL |

+-----------+-------------+-------------+--+

5 rows selected (0.167 seconds)

0: jdbc:hive2://cdh1:10000>

0: jdbc:hive2://cdh1:10000> select * from movierating limit 5;

INFO : Compiling command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d): select * from movierating limit 5

INFO : Semantic Analysis Completed

INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:movierating.userid, type:int, comment:null), FieldSchema(name:movierating.movieid, type:string, comment:null), FieldSchema(name:movierating.rating, type:tinyint, comment:null)], properties:null)

INFO : Completed compiling command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d); Time taken: 0.058 seconds

INFO : Executing command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d): select * from movierating limit 5

INFO : Completed executing command(queryId=hive_20180831151717_9c227dca-d789-4fa4-a247-b18daa66058d); Time taken: 0.01 seconds

INFO : OK

+---------------------+----------------------+---------------------+--+

| movierating.userid | movierating.movieid | movierating.rating |

+---------------------+----------------------+---------------------+--+

| NULL | NULL | NULL |

+---------------------+----------------------+---------------------+--+

5 rows selected (0.17 seconds)

0: jdbc:hive2://cdh1:10000>

其实，它们对应的HDFS上的文件是有数据的：

[hdfs@cdh3 ~]$ hdfs dfs -cat mysql_data/movielens/movie/part-m-00000 | head -n 11
1,Toy Story,1995
2,Jumanji,1995
3,Grumpier Old Men,1995
4,Waiting to Exhale,1995
5,Father of the Bride Part II,1995
6,Heat,1995
7,Sabrina,1995
8,Tom and Huck,1995
9,Sudden Death,1995
10,GoldenEye,1995
11,American President, The,1995
[hdfs@cdh3 ~]$ 
[hdfs@cdh3 ~]$ hdfs dfs -cat mysql_data/movielens/movierating/part-m-00000 | head -n 11
1,1193,5
1,661,3
1,914,3
1,3408,4
1,2355,5
1,1197,3
1,1287,5
1,2804,5
1,594,4
1,919,4
1,595,5
cat: Unable to write to output stream.
[hdfs@cdh3 ~]$

[hdfs@cdh3 ~]$ hdfs dfs -cat mysql_data/movielens/movie/part-m-00000 | head -n 11

1,Toy Story,1995

2,Jumanji,1995

3,Grumpier Old Men,1995

4,Waiting to Exhale,1995

5,Father of the Bride Part II,1995

6,Heat,1995

7,Sabrina,1995

8,Tom and Huck,1995

9,Sudden Death,1995

10,GoldenEye,1995

11,American President, The,1995

[hdfs@cdh3 ~]$

[hdfs@cdh3 ~]$ hdfs dfs -cat mysql_data/movielens/movierating/part-m-00000 | head -n 11

1,1193,5

1,661,3

1,914,3

1,3408,4

1,2355,5

1,1197,3

1,1287,5

1,2804,5

1,594,4

1,919,4

1,595,5

cat: Unable to write to output stream.

[hdfs@cdh3 ~]$

造成该问题的原因是H[……]

Cloudera, Cloudera CDH BEELINE, CDH, Cloudera, Hadoop, HIVE, NULL, 大数据, 数据表, 空值

HIVE，Beeline：Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near ‘stop’ ‘‘ ‘‘ (state=42000,code=40000)

Adamhuan 2018年8月31日 No Comments

在使用HIVE的beeline的时候你可能遇到如题所示的问题。
该问题发生在你希望退出HIVE的Beeline命令行的时候，具体如下：

[hdfs@cdh1 ~]$ beeline -u jdbc:hive2://cdh1:10000 -n hdfs
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
scan complete in 3ms
Connecting to jdbc:hive2://cdh1:10000
Connected to: Apache Hive (version 1.1.0-cdh5.10.0)
Driver: Hive JDBC (version 1.1.0-cdh5.10.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.10.0 by Apache Hive
0: jdbc:hive2://cdh1:10000> 
0: jdbc:hive2://cdh1:10000> exit;
Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near 'exit' '<EOF>' '<EOF>' (state=42000,code=40000)
0: jdbc:hive2://cdh1:10000> 
0: jdbc:hive2://cdh1:10000> quit;
Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near 'quit' '<EOF>' '<EOF>' (state=42000,code=40000)
0: jdbc:hive2://cdh1:10000>

[hdfs@cdh1 ~]$ beeline -u jdbc:hive2://cdh1:10000 -n hdfs

Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0

scan complete in 3ms

Connecting to jdbc:hive2://cdh1:10000

Connected to: Apache Hive (version 1.1.0-cdh5.10.0)

Driver: Hive JDBC (version 1.1.0-cdh5.10.0)

Transaction isolation: TRANSACTION_REPEATABLE_READ

Beeline version 1.1.0-cdh5.10.0 by Apache Hive

0: jdbc:hive2://cdh1:10000>

0: jdbc:hive2://cdh1:10000> exit;

Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near 'exit' '<EOF>' '<EOF>' (state=42000,code=40000)

0: jdbc:hive2://cdh1:10000>

0: jdbc:hive2://cdh1:10000> quit;

Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near 'quit' '<EOF>' '<EOF>' (state=42000,code=40000)

0: jdbc:hive2://cdh1:10000>

其实，正确的退出方法应该是这样的：
1：!quit
[crayon-5bc61c0c2accf7[……]

Cloudera CDH, Cloudera, code=40000), Error: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near 'stop' '' '' (state=42000, exit, Hadoop, HIVE, Linux, quit, 大数据, 退出

Cloudera CDH：机架的修改

Adamhuan 2018年8月28日 No Comments

在Cloudera中，默认情况下所有节点都会被分配到机架【/default】。

如果你修改了Cloudera集群中机架的分布，如下：

那么，在你执行过期重启之前，CDH的实际运行时的机架配置依旧会和以前的一样：

[cert@cdh1 ~]$ hdfs dfsadmin -printTopology
Rack: /default
   10.158.1.112:50010 (cdh2)
   10.158.1.113:50010 (cdh3)
   10.158.1.116:50010 (cdh4)
   10.158.1.122:50010 (cdh5)

[cert@cdh1 ~]$

[cert@cdh1 ~]$ hdfs dfsadmin -printTopology

Rack: /default

10.158.1.112:50010 (cdh2)

10.158.1.113:50010 (cdh3)

10.158.1.116:50010 (cdh4)

10.158.1.122:50010 (cdh5)

[cert@cdh1 ~]$

执行过期重启后，运行[……]

Cloudera, Cloudera CDH CDH, Cloudera, CM, Hadoop, hdfs dfsadmin -printTopology, Linux, modify, RACK, 修改, 命令行, 机架, 查看, 过期配置, 配置

Cloudera CDH – Hadoop + MySQL – Sqoop – Error：ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@54d9d12d is still active.

Adamhuan 2018年7月17日 No Comments

在使用Sqoop将MySQL的数据导入HDFS的时候你会遇到如题所示的错误。
具体如下：
1. 可能是这样的：

[hdfs@bigdata2 ~]$ sqoop import --connect jdbc:mysql://localhost/movielens --table movie --fields-terminated-by '\t' --username training --password *******
Warning: /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/07/17 19:20:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
18/07/17 19:20:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/07/17 19:20:13 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/07/17 19:20:13 INFO tool.CodeGenTool: Beginning code generation
18/07/17 19:20:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `movie` AS t LIMIT 1
18/07/17 19:20:14 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@54d9d12d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@54d9d12d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
        at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)
        at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524)
        at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)
        at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)
        at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:305)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
18/07/17 19:20:14 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1663)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

[hdfs@bigdata2 ~]$

[hdfs@bigdata2 ~]$ sqoop import --connect jdbc:mysql://localhost/movielens --table movie --fields-terminated-by '\t' --username training --password *******

Warning: /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

18/07/17 19:20:12 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3

18/07/17 19:20:12 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

18/07/17 19:20:13 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

18/07/17 19:20:13 INFO tool.CodeGenTool: Beginning code generation

18/07/17 19:20:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `movie` AS t LIMIT 1

18/07/17 19:20:14 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@54d9d12d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.

java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@54d9d12d is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)

at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)

at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)

at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)

at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)

at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)

at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)

at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524)

at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)

at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)

at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)

at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:305)

at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)

at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)

at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)

at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)

at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)

at org.apache.sqoop.Sqoop.run(Sqoop.java:147)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)

at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

18/07/17 19:20:14 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter

at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1663)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)

at org.apache.sqoop.Sqoop.run(Sqoop.java:147)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)

at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)

at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

[hdfs@bigdata2 ~]$

2. 也可能是这样的：

[hdfs@bigdata2 ~]$ sqoop import --connect jdbc:mysql://localhost/movielens --table movie --fields-terminated-by '\t' --username training --password ***** --driver=com.mysql.jdbc.driver
Warning: /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/07/17 19:34:39 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
18/07/17 19:34:39 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/07/17 19:34:39 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
18/07/17 19:34:39 INFO manager.SqlManager: Using default fetchSize of 1000
18/07/17 19:34:39 INFO tool.CodeGenTool: Beginning code generation
18/07/17 19:34:39 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.driver
        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875)
        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
[hdfs@bigdata2 ~]$

[hdfs@bigdata2 ~]$ sqoop import --connect jdbc:mysql://localhost/movielens --table movie --fields-terminated-by '\t' --username training --password ***** --driver=com.mysql.jdbc.driver

Warning: /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p0.2/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

18/07/17 19:34:39 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3

18/07/17 19:34:39 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

18/07/17 19:34:39 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.

18/07/17 19:34:39 INFO manager.SqlManager: Using default fetchSize of 1000

18/07/17 19:34:39 INFO tool.CodeGenTool: Beginning code generation

18/07/17 19:34:39 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.driver

java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.driver

at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875)

at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)

at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)

at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)