特殊字符问题：MQ消息输入错误：java。尼奥。查塞特。UnmappableCharacterException

Question

我有一个设置，其中有一个JMS生产者和JMS接收器。发送方应用程序发送如下消息：

source text ⟨е, ё, и, ю, я⟩  abcdefg

JMS接收器在接收到消息后，使用纯IBM MQ API类将其放入IBM MQ队列

将此消息发送到MQ时，我收到以下异常：

INFO  | 2020-09-17 09:45:19 | [main] mimq.MQReceiver (MQReceiver.java:211) - IO Exception Occurred: Input length = 1
java.nio.charset.UnmappableCharacterException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:282)
        at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.stringToBytes(JmqiCodepage.java:923)
        at com.ibm.mq.MQMessage.writeString(MQMessage.java:2848)
        at com.ibm.mimq.MQReceiver.sendToAnotherQueue(MQReceiver.java:192)
        at com.ibm.mimq.MQReceiver.main(MQReceiver.java:113)

下面是我的MQ PUT代码：

public static void sendToLocalQueue(String msg) {
    
    int port = 1414;
    String host = "some-host";
    String channel = "some-channel";
    String manager = "some-QM";
    String user = "user"; 
    String passwd = "passwd";
    String qname = "TEST";
    String qmname = "some-QM";
    
    MQQueueManager qMgr;
    MQQueue inputQ;
    
    try {       
        
        Hashtable<String, String> h = new Hashtable<String, String>();
        h.put(MQC.TRANSPORT_PROPERTY, MQC.TRANSPORT_MQSERIES_CLIENT);
        MQEnvironment.properties = h;
        MQEnvironment.hostname  = host;
        MQEnvironment.port      = port;
        MQEnvironment.channel   = channel;
        
        MQEnvironment.userID    = user;
        MQEnvironment.password  = passwd;
        MQEnvironment.disableTracing();
        MQException.log = null;
        qMgr = new MQQueueManager(manager);
        
        MQMessage m = new MQMessage();      
        m.applicationOriginData = "AMPS";
        m.messageType   = MQC.MQMT_DATAGRAM;
        m.format        = MQC.MQFMT_STRING;
        m.encoding      = MQC.MQENC_NATIVE;
        m.priority      = 4;
        m.persistence   = MQC.MQPER_PERSISTENT;
        m.characterSet  = MQC.MQCCSI_Q_MGR;
        //m.characterSet = 1208;
        m.expiry        =  MQC.MQEI_UNLIMITED;
        m.writeString(msg);
        
        MQPutMessageOptions putOptions = new MQPutMessageOptions();
        putOptions.options = MQC.MQPMO_SYNCPOINT | MQC.MQPMO_FAIL_IF_QUIESCING; 
        
        logger.info("Putting message to LAN MQ (TEST queue)....");          
        qMgr.put(qname, qmname, m, putOptions);
        qMgr.commit();
        
        
    } catch(MQException me) {
        logger.info("Error Code       : "  +me.getErrorCode());
        logger.info("LocalizedMessage : "  +me.getLocalizedMessage());
        logger.info("Message          : "  +me.getMessage());
        logger.info("Reason           : "  +me.getReason());
        me.printStackTrace();
        
    } catch (IOException e) {
        // TODO Auto-generated catch block
        logger.info("IO Exception Occurred       : "  +e.getLocalizedMessage());
        e.printStackTrace();
    }       
}

由于无法映射字符，无法将消息放入队列。在队列管理器级别，编码设置为UTF-8

然而，当我替换下一行时：m.characterSet = MQC.MQCCSI_Q_MGR;

有了这句话：m.characterSet = 1208;问题就不存在了

我的问题是为什么这种转换不能在MQ级别完成。我需要检查哪些设置以确保正确的转换。我尝试过以下技巧，但不起作用：

Setting java parameter as : -Dfile.encoding=UTF-8 

我的环境：

Server : Linux
MQ : 9.0  or 7.5
Java : 1.8

还有一件事要提的是，相同的设置在7.5上工作，但在迁移后没有在MQ 9.0上工作。我知道，通过我上面一行代码的更改，我可以传递信息。但我想在MQ级别了解一下，我是否遗漏了一些配置。任何建议都将不胜感激

谢谢

更新

我从中向MQ发送消息的客户机具有CCSID:MQMD.CodedCharSetId = 1208

我连接并发送消息的MQ服务器具有以下功能：

getDefaultProperty(Object) returns [819(0x333)] Integer
setCCSID(int) setter [819(0x333)]

因此，当我在代码中明确设置1208时，它是有效的。当转换失败时

更新-2

我在罐子里看到的值MQC.MQCCSI_Q_MGR is Zero。因此，代码是这样设计的，如果值为零，它将从设置为819的Jar中获取默认值。我在打开MQ跟踪时了解到了这一点。代码是这样的：

getDefaultProperty(Object) returns [819(0x333)] Integer
setCCSID(int) setter [819(0x333)]

这段代码存在于jar中。所以我们需要在消息上显式设置字符集值。就我而言是1208

Answer 1

当本地连接到队列管理器（即使用TRANSPORT_MQSERIES_BINDINGS）时，代码：-

m.characterSet     = MQC.MQCCSI_Q_MGR;

表示“获取队列管理器属性CCSID中设置的CCSID”。您可以使用以下MQSC命令查看此属性：-

DISPLAY QMGR CCSID

当作为客户机连接时（正如您的代码所示），则代码：-

m.characterSet     = MQC.MQCCSI_Q_MGR;

表示“从客户端计算机区域设置中查找CCSID”

IBM Knowledge Center表示：

For client applications, MQCCSI_Q_MGR is filled in, based on the locale of the client rather than the one on the queue manager.

如果将代码行更改为显式地将消息CCSID设置为1208，则客户端计算机区域设置似乎没有设置为UTF-8解决了问题

通过浏览队列上的消息（不进行转换）并查看MQMD.CodedCharSetId字段中的内容，可以查看客户端设置的内容

Answer 2

这在MQ v7上起作用的原因。5，而不是MQ v9。0是因为在IBM MQ v8之前。0使用java.nio.charset.Charset.encode(CharBuffer)为Java编码的数据创建IBM MQ类，这会导致错误或不可翻译数据的默认替换。由于默认的characterSet是819（ASCII），这将导致您发送的任何无法转换为ASCII的字符被透明地替换为默认替换字符，在大多数情况下，这意味着数据被替换为?字符

v8之后。0默认行为已更改，将此情况报告为错误，默认情况下不再替换格式错误或不可翻译的数据

---

将字符集设置为UTF-8的解决方案是最好的解决方案，因为这会导致发送您想要发送的确切数据

另一个选项是告诉MQ使用之前的行为

---

IBM MQ 9.0知识中心页面Developing applications>Developing JMS and Java applications>Using IBM MQ classes for Java>Character string conversions in IBM MQ classes for Java中描述了新行为的描述，以及如何配置IBM MQ Class for Java以使用以前的行为：

From IBM® MQ Version 8.0, some of the default behavior regarding character string conversion in the IBM MQ classes for Java™ has changed.

Before IBM MQ Version 8.0, string conversions in IBM MQ classes for Java was done by calling the java.nio.charset.Charset.decode(ByteBuffer) and Charset.encode(CharBuffer) methods.

Using either of these methods results in a default replacement ( REPLACE) of malformed or untranslatable data.

This behavior can obscure errors in applications, and lead to unexpected characters, for example ?, in translated data. From IBM MQ Version 8.0, to detect such issues earlier and more effectively, the IBM MQ classes for Java use CharsetEncoders and CharsetDecoders directly and configure the handling of malformed and untranslatable data explicitly.

From IBM MQ Version 8.0, the default behavior is to REPORT such issues by throwing a suitable MQException.

...

Setting system defaults

From IBM MQ Version 8.0, the following two Java system properties are available to configure default behavior regarding character string conversion.

com.ibm.mq.cfg.jmqi.UnmappableCharacterAction Specifies the action to be taken for untranslatable data on encoding and decoding. The value can be REPORT, REPLACE, or IGNORE.

com.ibm.mq.cfg.jmqi.UnmappableCharacterReplacement Sets or gets the replacement bytes to apply when a character cannot be mapped in an encoding operation The default Java replacement string is used in decoding operations.

To avoid confusion between Java character and native byte representations, you should specify com.ibm.mq.cfg.jmqi.UnmappableCharacterReplacement as a decimal number representing the replacement byte in the native character set.

For example, the decimal value of ?, as a native byte, is 63 if the native character set is ASCII-based, such as ISO-8859-1, while it is 111 if the native character set is EBCDIC.

---

如果要模拟先前的行为，可以设置以下系统属性：

-Dcom.ibm.mq.cfg.jmqi.UnmappableCharacterAction=REPLACE
-Dcom.ibm.mq.cfg.jmqi.UnmappableCharacterReplacement=63

您还可以通过以下方式进行编程设置：

System.setProperty("com.ibm.mq.cfg.jmqi.UnmappableCharacterAction", "REPLACE");
System.setProperty("com.ibm.mq.cfg.jmqi.UnmappableCharacterReplacement", "63");