devbox
酷酷是懒虫
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

import java.io后报错,遇到IOException异常运行导入作业：由javac的返回错误：java.io.IOException异常...

作者：酷酷是懒虫 | 2024-06-22 13:59:04

赞

踩

import failed: java.io.ioexception: error returned by javac at org.apache.s

I am trying to run a simple sqoop import program using JAVA.

My Program:

String driver="com.vertica.Driver";

Configuration config = new Configuration();

config.addResource(new Path("/../../../mapred-site.xml"));

config.addResource(new Path("/../../../core-site.xml"));

config.addResource(new Path("/../../../hdfs-site.xml"));

SqoopOptions options = new SqoopOptions();

options.setConnectString(connection_string);

options.setDriverClassName(driver);

options.setUsername(username);

options.setPassword("xxxxxxxxxxxxx");

options.setTargetDir(directory path);

options.setFetchSize(0);

options.setSqlQuery("SELECT 123456 as runid, " +

"business_date " +

"FROM database.table " +

"WHERE business_date=\'20140922\' " +

"AND instrument_id=1037894110378941 " +

"AND $CONDITIONS");

int ret = new ImportTool().run(options);

System.out.println("Return Val: "+ret);

I have the following error

15/02/03 02:26:23 WARN sqoop.ConnFactory: $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.

15/02/03 02:26:23 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.

15/02/03 02:26:23 INFO tool.CodeGenTool: Beginning code generation

15/02/03 02:26:24 INFO manager.SqlManager: Executing SQL statement: SELECT 123456 as runid, business_date FROM silverscreen_qa.SS_INSTRUMENT WHERE business_date='20140922' AND instrument_id=1037894110378941 AND (1 = 0)

15/02/03 02:26:24 INFO manager.SqlManager: Executing SQL statement: SELECT 123456 as runid, business_date FROM silverscreen_qa.SS_INSTRUMENT WHERE business_date='20140922' AND instrument_id=1037894110378941 AND (1 = 0)

15/02/03 02:26:24 INFO orm.CompilationManager: $HADOOP_MAPRED_HOME is not set

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:11: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.JdbcWritableBridge;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:12: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.DelimiterSet;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:13: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.FieldFormatter;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:14: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.RecordParser;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:15: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.BooleanParser;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:16: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.BlobRef;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:17: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.ClobRef;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:18: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.LargeObjectLoader;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:19: error: package com.cloudera.sqoop.lib does not exist

import com.cloudera.sqoop.lib.SqoopRecord;

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:37: error: cannot find symbol

public class QueryResult extends SqoopRecord implements DBWritable, Writable {

^

symbol: class SqoopRecord

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:81: error: cannot find symbol

public void loadLargeObjects(LargeObjectLoader __loader)

^

symbol: class LargeObjectLoader

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:119: error: cannot find symbol

private final DelimiterSet __outputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:123: error: cannot find symbol

public String toString(DelimiterSet delimiters) {

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:129: error: cannot find symbol

public String toString(DelimiterSet delimiters, boolean useRecordDelim) {

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:140: error: cannot find symbol

private final DelimiterSet __inputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:141: error: cannot find symbol

private RecordParser __parser;

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:142: error: package RecordParser does not exist

public void parse(Text __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:150: error: package RecordParser does not exist

public void parse(CharSequence __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:158: error: package RecordParser does not exist

public void parse(byte [] __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:166: error: package RecordParser does not exist

public void parse(char [] __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:174: error: package RecordParser does not exist

public void parse(ByteBuffer __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:182: error: package RecordParser does not exist

public void parse(CharBuffer __record) throws RecordParser.ParseError {

^

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:64: error: bad operand types for binary operator '=='

if (this == o) {

^

first type: QueryResult

second type: Object

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:67: error: inconvertible types

if (!(o instanceof QueryResult)) {

^

required: QueryResult

found: Object

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:70: error: inconvertible types

QueryResult that = (QueryResult) o;

^

required: QueryResult

found: Object

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:78: error: cannot find symbol

this.runid = JdbcWritableBridge.readLong(1, __dbResults);

^

symbol: variable JdbcWritableBridge

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:79: error: cannot find symbol

this.business_date = JdbcWritableBridge.readDate(2, __dbResults);

^

symbol: variable JdbcWritableBridge

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:89: error: cannot find symbol

JdbcWritableBridge.writeLong(runid, 1 + __off, -5, __dbStmt);

^

symbol: variable JdbcWritableBridge

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:90: error: cannot find symbol

JdbcWritableBridge.writeDate(business_date, 2 + __off, 91, __dbStmt);

^

symbol: variable JdbcWritableBridge

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:119: error: cannot find symbol

private final DelimiterSet __outputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:132: error: cannot find symbol

__sb.append(FieldFormatter.escapeAndEnclose(runid==null?"null":"" + runid, delimiters));

^

symbol: variable FieldFormatter

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:134: error: cannot find symbol

__sb.append(FieldFormatter.escapeAndEnclose(business_date==null?"null":"" + business_date, delimiters));

^

symbol: variable FieldFormatter

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:140: error: cannot find symbol

private final DelimiterSet __inputDelimiters = new DelimiterSet((char) 44, (char) 10, (char) 0, (char) 0, false);

^

symbol: class DelimiterSet

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:144: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:152: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:160: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:168: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:176: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:184: error: cannot find symbol

this.__parser = new RecordParser(__inputDelimiters);

^

symbol: class RecordParser

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:206: error: cannot find symbol

QueryResult o = (QueryResult) super.clone();

^

symbol: variable super

location: class QueryResult

/tmp/sqoop-sha0_dev/compile/bda7671a1a4e5e172248f2497654d2c1/QueryResult.java:208: error: incompatible types

return o;

^

required: Object

found: QueryResult

41 errors

15/02/03 02:26:24 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Error returned by javac

at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:207)

at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:97)

at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:396)

at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)

at com.cts.rbc.archive.TestConnectionForHive.main(TestConnectionForHive.java:196)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Return Val: 1

Possible Solutions I tried.

In my hadoop-env.sh the value of JAVA_HOME is

JAVA_HOME=/usr/lib/jvm/java-1.7.0-oracle-1.7.0.65.x86_64

I know this sounds repeated but believe me I have tried all solutions described in the solutions provided in similar questions

I tried changing JAVA_HOME in ~/.bashrc with the below code

JAVA_HOME=/etc/alternatives/java

This is where latest java is installed in my node

It still throws the same error.

The same query works in bash.

Note: I don't have permission to alter JAVA_HOME in hadoop-env.sh as its a shared system.

Possible way to overwrite JAVA_HOME in Java Program?

解决方案

[solved]

There was no need to change java_home (or) hadoop_mapred_home (or) sqoop_home

As the error suggests, it was not able to find sqoop library. Whereas I used sqoop 1.4.4 library in my program.

Please note that I wasn't using any build tool (Maven, SBT or anything)

I was using normal eclipse and adding external .jar to my program.

While exporting the .jar I was using Eclipse standard method (File -> Export -> Java -> Runnable .jar File)

In launch configuration I selected my appropriate project

and in library handling, I was using the option "package required libraries into generated JAR"

When I selected "extract required libraries into generated JAR" and executed .jar file it was executed successfully.

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/酷酷是懒虫/article/detail/746708
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号