EMQ X Enables bidirectional SSL/TLS secure connections
preface
This paper is mainly divided into two parts, one, how to achieve the function, two, encountered problems and solutions.
It took a day of stumbling through this “seemingly simple requirement”, but it was actually a very simple requirement.
In this process, I found that I could not find a solution by searching on the Internet, so I started to record the solution process. On the one hand, I can review it later, and on the other hand, I hope to give an idea to people who encounter similar problems later.
SSL configuration for EMQX
Looking at the configuration file, the default port for MQTT TLS is 8883:
listener.ssl.external = 8883
Copy the codeYou need to configure the server certificate and CA in the /etc/emqx/emqx.conf file.
listener.ssl.external.keyfile = etc/certs/key.pem
listener.ssl.external.certfile = etc/certs/cert.pem
listener.ssl.external.cacertfile = etc/certs/cacert.pem
Copy the codeThe default etc/certs directory is the self-signed certificate generated by EMQ X
Pem, cert.pem, and cacert.pem are configured on the EMQX server.
Pem, client-cert.pem, and cacert.pem are configured on the client
This default certificate is only for testing purposes, we need to generate our own certificate for actual use
The cipher list supported by the server must be explicitly specified. The default cipher list is the same as Mozilla’s server cipher list:
listener.ssl.external.ciphers = ECDHE-ECDSA-AES256-GCM-SHA384,ECDHE-RSA-AES256-GCM-SHA384...
Copy the codeHow do I establish an SSL connection in Java
Connection configuration
-
The lower port number needs to be changed. The direct connection port is 1883, and the SSL port is 8883.
-
An SSL link factory resolution certificate is required in the link option.
public void createClient(a) throws Exception {
try {
/ / create the client
// Address of the broker EMq, clientId current service name (unique), MqttClientPersistence, cache contents during message transmission
client = new MqttClient(mqtt.getBroker(), mqtt.getClientId(), new MemoryPersistence());
// MQTT connection options
MqttConnectOptions connOpts = new MqttConnectOptions();
connOpts.setUserName(mqtt.getUsername());
connOpts.setPassword(mqtt.getPassword().toCharArray());
// Clear the session
connOpts.setCleanSession(true);
// Heartbeat interval
connOpts.setKeepAliveInterval(180);
connOpts.setAutomaticReconnect(true);
// SSL connection, certificate configuration
SSLSocketFactory factory = EmqxSSLFactory.createSocketFactory(
ssl.getCaPath(), / / ca address
ssl.getCertPath(), / / address cert
ssl.getKeyPath(), / / key address
ssl.getPassword()); // Password (no password "")
connOpts.setSocketFactory(factory);
// Create a link
client.connect(connOpts);
// Set the callback
client.setCallback(new OnMessageCallback(client, topicListeners));
// Subscribe messages for the first time
for(TopicListener topicListener : topicListeners) { client.subscribe(topicListener.getTopic(), topicListener); }}catch (MqttException me) {
log.error("reason:{} ", me.getReasonCode());
log.error("msg {}", me.getMessage());
log.error("loc {}", me.getLocalizedMessage());
log.error("cause :{}", JSONUtil.toJsonStr(me.getCause()));
log.error("exception :{}", JSONUtil.toJsonStr(me));
me.printStackTrace();
throwme; }}Copy the codeAdd POM dependencies
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.47</version>
</dependency>
Copy the codeAnalytical certificate
The code to parse the certificate is extracted from mqtt.fx’s client unjar package, and is essentially the same as the code I found online. The reason is a later problem.)
public static SSLSocketFactory createSocketFactory(String caCertFile, String clientCertFile, String privateKeyFile, String password) throws Exception {
Security.addProvider(new BouncyCastleProvider());
X509Certificate caCert = parseCert(caCertFile);
X509Certificate clientCert = parseCert(clientCertFile);
PemReader pemReader = new PemReader(new FileReader(privateKeyFile));
byte[] content = pemReader.readPemObject().getContent();
PKCS8EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(content);
PrivateKey privateKey = KeyFactory.getInstance("RSA").generatePrivate(privateKeySpec);
KeyStore caKs = KeyStore.getInstance(KeyStore.getDefaultType());
caKs.load((InputStream) null, (char[]) null);
caKs.setCertificateEntry("ca-certificate", caCert);
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(caKs);
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
ks.load((InputStream) null, (char[]) null);
ks.setCertificateEntry("certificate", clientCert);
ks.setKeyEntry("private-key", privateKey, password.toCharArray(), new java.security.cert.Certificate[]{clientCert});
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(ks, password.toCharArray());
SSLContext context = SSLContext.getInstance("TLSv1.2");
context.init(kmf.getKeyManagers(), tmf.getTrustManagers(), (SecureRandom) null);
return context.getSocketFactory();
}
public static X509Certificate parseCert(String certPath) throws Exception {
CertificateFactory cf = CertificateFactory.getInstance("X.509");
InputStream inStream = new FileInputStream(certPath);
return (X509Certificate) cf.generateCertificate(inStream);
}
Copy the codeTheoretically, following the documentation, such a simple configuration can then be directly linked.
However… Life is always up and down, down, down, down, down, down, down, down, down.
Problems encountered
Fault 1: The trust IP address is not configured in the official default CA certificate
Reported an exception is the Java security. Cert. CertificateException: No subject the alternative names present, literal translation is that there is No subject alternative name.
Stack information is as follows:
Caused by: java.security.cert.CertificateException: No subject alternative names present
at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:156) ~[na:1.8. 0 _282]
at sun.security.util.HostnameChecker.match(HostnameChecker.java:100) ~[na:1.8. 0 _282]
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:457) ~[na:1.8. 0 _282]
at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:417) ~[na:1.8. 0 _282]
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:230) ~[na:1.8. 0 _282]
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129) ~[na:1.8. 0 _282]
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638) ~[na:1.8. 0 _282]
Copy the codeFirst, I looked at the stack information and found that the exception was in matchIp and the subjAltNames was empty. Look at the for loop below and what it does is it takes the IP from this list and matches it.
private static void matchIP(String expectedIP, X509Certificate cert)
throws CertificateException { Collection<List<? >> subjAltNames = cert.getSubjectAlternativeNames();if (subjAltNames == null) {
throw new CertificateException
("No subject alternative names present");
}
for(List<? > next : subjAltNames) {// For IP address, it needs to be exact match
if (((Integer)next.get(0)).intValue() == ALTNAME_IP) {
String ipAddress = (String)next.get(1);
if (expectedIP.equalsIgnoreCase(ipAddress)) {
return;
} else {
// compare InetAddress objects in order to ensure
// equality between a long IPv6 address and its
// abbreviated form.
try {
if (InetAddress.getByName(expectedIP).equals(
InetAddress.getByName(ipAddress))) {
return; }}catch (UnknownHostException e) {
} catch (SecurityException e) {}
}
}
}
throw new CertificateException("No subject alternative " +
"names matching " + "IP address " +
expectedIP + " found");
}
Copy the codeThen I looked up the SSL data and found that the certificate has a subjectAltName field. We can add IP/domain to the trusted domain list by subjectAltName =127.0.0.1 / DNS = localhost. Multiple IP addresses and DNS servers can be configured. Then I also opened the default cacert.pem and saw that there was no configuration.
Context.init (kmf.getKeyManagers(), tmf.getTrustManagers(), (SecureRandom) null); As you can see, the trustMagager is fetched from TMF, which is initialized with caCert.
The conclusion is that the 127.0.0.1 that I want to access is not a trusted domain name in ca.pem. So I need to manually generate a certificate, should be no problem. Nice, _,!
In actual development, I initially validated 127.0.0.1 with the MQTTX client tool and found that 127.0.0.1 was untrusted, then validated 127.0.0.1 with the MQTT.fx client tool and found that it was ok. So at first I thought I had written the code wrong, so I was confused and didn’t think in the direction of the wrong certificate.
After discovering that MQtt.fx was written in Java, I unpacked the JAR and found that it linked the same way I had, so I assumed that the client had bypassed the verification of trusted domain names, or that 127.0.0.1 was local and trusted by default. Since I did not find the source code, only the class file, I did not look at the relevant code in more detail.
Problem 2: File access permission during certificate generation
Since I knew it was a certificate problem, I searched the method of generating a certificate using OpenSSL EMQ X enable bidirectional SSL/TLS secure connection
Then follow the steps above to generate the correct certificate. Then repeat the steps in the morning, configure the EMQX certificate, configure the client certificate, and link again.
A new error was found
- The emQX server directly disconnects the authentication and the connection fails.
- The mqtt.fx client logs to check the validity of the certificate and password.
Seeing these error messages, I thought it was my certificate generation error, and then went to the Internet to search for other certificate generation methods, the generation methods are almost the same, of course, the results are similar, are failure, error.
After a day of agonizing, I repeatedly checked the EMQX document and found that the debug mode could be enabled and detailed logs could be viewed.
You can enable logs in either of the following ways:
/etc/emqx/emqx.confTo configure log levelslog.level = debugAnd then restart- The command line
emqx_ctl log set-level <Level>To dynamically change the log level at run time
The log output file path can be found in emqx.conf
log.to = file log.dir = /var/log/emqx log.file = emqx.log
Retry connection found, log output
2021-06-23T16:12:01.512814+08:00 [error] Supervisor: 'esockD_Connection_sup - <0.1861.0>', errorContext: connection_shutdown , reason: {ssl_error,{options,{keyfile,"/etc/emqx/certs/emqx.key",{error,eacces}}}}, offender: [{pid, < 0.2049.0 >}, {name, connection}, {{mfargs, emqx_connection start_link, [[{deflate_options,[]},{max_conn_rate,500},{active_n,100},{zone,external} {proxy_address_header,<<>>},{proxy_port_header,<<>>},{supported_subprotocols,[]}]]}}]Copy the codeThe emqx.key file {errror, eacces} is found according to the information. I am familiar with this file. In Linux environment, I do not use sudo, and often encounter insufficient file permissions.
I’m using Sudo emqx start, which should be able to read any file. Pem and emqx.pem are read-only files. Emqx. key is the only key file that cannot be read or written. , _,?
Then modify the file to be readable and try to connect again. The connection succeeds.
over
Reference documentation
- Emq official document
- EMQ X Enables bidirectional SSL/TLS secure connections