运行python编写的配置单元udf时的问题

Question

我用python编写了一个简单的hive udf，但是当我在hive shell中运行它时，会抛出以下错误：

Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Hive Runtime Error while closing operators
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
        at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:514)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
        ... 8 more


FAILED: Execution Error, return code 20003 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. An error occurred when trying to close the Operator running your custom script.

`

配置单元外壳中使用的命令：

add file /path/to/mycode.py;

创建表

加载数据

SELECT TRANSFORM (fname,lname) USING 'python mycode.py' AS (fname,LNAME) from table;

Hadoop版本：2.7

蜂箱：0.13

霉菌素.py

^{pr2}$

错误：

(<type 'exceptions.ValueError'>, ValueError('need more than 1 value to unpack',), <traceback object at 0x7f3441c14050>) NULL

不过，当我尝试cat pyinp.txt | python mycode.py时，它会给出所需的输出。在

有人能帮我解决这个问题吗？在

Answer 1

这就解决了其他情况下的问题：here

他们在函数中使用try-catch块，你也这么做了吗？在