HTTPS is a secure HTTP channel, which is simply the secure version of HTTP. That is, add SSL layer to HTTP, and SECURE Sockets Layer (SSL) is the basis of HTTPS security. Therefore, SSL is required for details of encryption. Nebula is a high-performance event-driven networking framework for developers to quickly develop high-concurrency Web services or build clusters of high-concurrency distributed services. Nebula’s HTTPS support as a generic networking framework is important, and Nebula can be used as both an HTTPS server and an HTTPS client. This article takes a detailed look at OPENSSL-based SSL programming with the HTTPS implementation of the Nebula framework. If you find this article useful with Nebula’s Github or code cloud, give it to Star. Thank you. Nebula is not only a framework, but also provides a series of applications built on it with the goal of creating a high-performance distributed service cluster solution. Nebula’s main application areas: instant messaging (a success with Nebula for an IM), a messaging platform, real-time analytics and computing (a success story), and Bwar plans to develop crawler applications based on Nebula.

1. SSL encryption communication

HTTPS communication adds an SSL layer between the TCP communication layer and the HTTP application layer. If the application layer is not HTTP, SSL can be used to encrypt communication, such as the WSS of WebSocket protocol WS plus the SSL layer. With Nebula’s ADDITION of SSL support, all Nebula supported protocols now have SSL encryption support, leaving no modifications to the Nebula business code.

SSL connection establishment process after Socket connection establishment:

2. OpenSSL API

There are many OpenSSL apis, but not all of them are used. You can read the API documentation if you want to see how to use an API in detail.

2.1 Initializing OpenSSL

OpenSSL must be initialized before it can be used. Before establishing an SSL connection, specify the protocol and version used for the connection for the Client and Server respectively. Currently, the protocol versions available include SSLv2, SSLv3, SSLv2/v3, and TLSv1.0. To establish an SSL connection, the Client and Server must use mutually compatible protocols. Below is the Nebula framework SocketChannelSslImpl: : SslInit () function to initialize the OpenSSL code, according to the different versions of OpenSSL calls the initialized different apis.

#if OPENSSL_VERSION_NUMBER >= 0x10100003L if (OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL) == 0) { pLogger->WriteLog(neb::Logger::ERROR, __FILE__, __LINE__, __FUNCTION__, "OPENSSL_init_ssl() failed!" ); return(ERR_SSL_INIT); } /* * OPENSSL_init_ssl() may leave errors in the error queue * while returning success */ ERR_clear_error(); #else OPENSSL_config(NULL); SSL_library_init(); SSL_load_error_strings(); SSL_load_error_strings(); SSL_load_error_strings(); // Initialize error messages OpenSSL_add_all_algorithms(); #endifCopy the code

2.2 create CTX

CTX is an SSL session environment. The CTX varies according to the protocol used for establishing connections. Create OpenSSL functions for CTX:

SSL_CTX_new(); SSL_CTX_new(); SSL_CTX_set_verify(); SSL_CTX_set_verify(); SSL_CTX_load_verify_location(); Int SSL_CTX_use_certificate_file(); // Load the list of CA certificates trusted by the application for the SSL session. Int SSL_CTX_use_certificate_chain_file(); Int SSL_CTX_use_PrivateKey_file(); int SSL_CTX_use_PrivateKey_file(); SSL_CTX_check_private_key(); SSL_CTX_check_private_key(); // Verify that the loaded private key matches the certificateCopy the code

2.3 Creating an SSL Socket

Before creating an SSL Socket, you need to create a Socket to establish a TCP connection. Create SSL socket related functions:

SSL *SSl_new(SSL_CTX *ctx); SSL_set_fd(SSL * SSL, int fd); SSL_set_rfd(SSL * SSL, int fd); SSL_set_wfd(SSL * SSL, int fd); // Bind the stream socket in write only modeCopy the code

2.4 Completing the SSL Handshake

In this step, we need to establish an SSL connection on the basis of the ordinary TCP connection. The procedure for establishing a connection with a normal stream socket is similar: The Client initiates the handshake using the function SSL_connect(), which is similar to connect() in a stream socket, while the Server responds to the handshake using the function ssl_accept (), which is similar to Accept () in a stream socket. The two functions are prototyped as follows:

int SSL_connect(SSL *ssl);
int SSL_accept(SSL *ssl);
Copy the code

After the handshake is complete, the Client usually asks the Server to send certificate information for authentication. The implementation uses the following two functions:

X509 *SSL_get_peer_certificate(SSL *ssl); X509_NAME *X509_get_subject_name(X509 *a); // The name of the person used to get the certificateCopy the code

2.5 Data Transmission

After the previous series of procedures, you are ready for secure data transfer. SSL_read() and SSL_write() are used to replace the read() and write() functions used by ordinary stream sockets to read and write SSL sockets during data transfer. The two new functions are modeled as follows:

int SSL_read(SSL *ssl,void *buf,int num); Int SSL_write(SSL * SSL,const void *buf,int num); // Write data to the SSL socketCopy the code

2.6 Session End

When the communication between Client and Server is complete, the following functions are used to release the SSL resources requested in the previous procedure:

int SSL_shutdown(SSL *ssl); Void SSl_free(SSL * SSL); Void SSL_CTX_free(SSL_CTX * CTX); // Release the SSL session environmentCopy the code

3. The SSL and TLS

HTTPS uses the Secure Socket Layer (SSL) and Transport LayerSecurity (TLS) protocols. SSL technology was pioneered by browser developer Netscape Communications, which developed versions prior to SSL3.0. At present, the initiative has been transferred to the Internet Engineering Task Force (IETF).

IETF developed TLS1.0, TLS1.1 and TLS1.2 based on SSL3.0. TSL is based on SSL and is sometimes referred to as SSL. The current mainstream versions are SSL3.0 and TLS1.0.

The SSL1.0 protocol was never put into use because problems were found at the beginning of its design. SSL2.0 was also found to have problems, so many browsers simply scrapped the protocol version.

4. The SSL communication implementation in Nebula

The Nebula framework supports both SSL server and SSL client applications, and openSSL is initialized only once (SslInit() is called only once). The Nebula framework’s SSL-related code, including client and server implementations, is encapsulated in the SocketChannelSslImpl class. Nebula’s SSL communication is based on asynchronous, non-blocking socket communication and does not use OpenSSL’s BIO (the code is more complex because it is not necessary).

SocketChannelSslImpl is a descendant of SocketChannelImpl, which adds an SSL communication layer on top of SocketChannelImpl regular TCP communication. The calls to the two classes are almost identical. The SocketChannelSslImpl class is declared as follows:

class SocketChannelSslImpl : public SocketChannelImpl { public: SocketChannelSslImpl(SocketChannel* pSocketChannel, std::shared_ptr<NetLogger> pLogger, int iFd, uint32 ulSeq, Ev_tstamp dKeepAlive = 0.0). virtual ~SocketChannelSslImpl(); static int SslInit(std::shared_ptr<NetLogger> pLogger); static int SslServerCtxCreate(std::shared_ptr<NetLogger> pLogger); static int SslServerCertificate(std::shared_ptr<NetLogger> pLogger, const std::string& strCertFile, const std::string& strKeyFile); static void SslFree(); int SslClientCtxCreate(); int SslCreateConnection(); int SslHandshake(); int SslShutdown(); virtual bool Init(E_CODEC_TYPE eCodecType, bool bIsClient = false) override; // Override the Send() method of the base class to continue the establishment of SSL connections after the establishment of non-blocking socket connections and Send and receive data. virtual E_CODEC_STATUS Send(int32 iCmd, uint32 uiSeq, const MsgBody& oMsgBody) override; virtual E_CODEC_STATUS Send(const HttpMsg& oHttpMsg, uint32 ulStepSeq) override; virtual E_CODEC_STATUS Recv(MsgHead& oMsgHead, MsgBody& oMsgBody) override; virtual E_CODEC_STATUS Recv(HttpMsg& oHttpMsg) override; virtual E_CODEC_STATUS Recv(MsgHead& oMsgHead, MsgBody& oMsgBody, HttpMsg& oHttpMsg) override; virtual bool Close() override; protected: virtual int Write(CBuffer* pBuff, int& iErrno) override; virtual int Read(CBuffer* pBuff, int& iErrno) override; private: E_SSL_CHANNEL_STATUS m_eSslChannelStatus; // Add SSL channel status to base m_ucChannelStatus channel status bool m_bIsClientConnection; SSL* m_pSslConnection; static SSL_CTX* m_pServerSslCtx; When the SSL option is turned on and the Nebula service is started, static SSL_CTX* m_pClientSslCtx is automatically created; Void by default, created when the SSL option is turned on and a connection to another SSL service is first initiated (such as accessing an HTTPS address)};Copy the code

SocketChannelSslImpl methods with override are methods of the same name that override the base SocketChannelImpl and are key to transparent calls for SSL and non-SSL communications. Methods that do not contain the override keyword are SSL communication-related methods that contain openSSL function calls. Methods without Override are static and non-static. Static methods are called only once in a process, regardless of the Channel object. There is no need to call non-static SSL-related methods outside of SocketChannel.

SSL_do_handshake(), SSL_write(), SSL_read(), and SSL_get_error() do not provide an error code because they are non-blocking sockets. Both SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE are normal.

Most of the openSSL sample programs on the web simply implement synchronous SSL communication by calling openSSL functions sequentially. In non-blocking IO applications, SSL communication is much more complicated. SocketChannelSslImpl implements non-blocking SSL communication. The whole communication process is not completely linear from the implementation of this class. The FOLLOWING SSL communication diagram illustrates more clearly how SSL communication is implemented in the Nebula framework:

Static methods in SocketChannelSslImpl only need to be called once during the lifetime of the process, or SSL_CTX_new(), SSL_CTX_free(), and so on. Further understanding that the SSL_CTX structure only needs to be created once in a process (one for each Server and Client in Nebula) to be used by all SSL connections; Of course, it’s okay to create a separate SSL_CTX for each SSL connection (Nebula 0.4 tested creating a separate SSL_CTX for each Client), but it’s not usually done because it consumes more memory and is less efficient.

To establish an SSL connection, the client calls SSL_connect() and the server calls SSL_accept(), as many OpenSSL demos use. Nebula uses SSL_do_handshake(), which is suitable for both client and server services, and SSL_do_handshake() for services that combine client and server functions. Notice Call SSL_set_connect_state() on the client and SSL_set_accept_state() on the server before calling SSL_do_handshake(). In non-blocking IO, SSL_do_handshake() may require multiple calls to complete the handshake, SSL_ERROR_WANT_READ and SSL_ERROR_WANT_WRITE determine whether to listen for read or write events based on SSL_get_error(). Call SSL_do_handshake() when the corresponding event is triggered. For details, see SocketChannelSslImpl’s Send and Recv methods.

To close the SSL connection, call SSL_shutdown() to close the SSL connection (SSL_shutdown() may be called several times in non-blocking I/OS), call SSL_free() to release the SSL connection resources, and close the socket connection. SSL_CTX does not need to be released. With Nebula 0.4’s SSL client and SSL Server functions in a loop with curl on multiple terminals:

while : do curl -v -k -H "Content-Type:application/json" -X POST -d '{"hello":"nebula ssl test"}' https://192.168.157.168:16003/test_ssl the doneCopy the code

The test method is shown below:

The memory usage of the SSL Server keeps increasing, which is suspected to be a memory leak. However, pmap -d checks that the anon memory of a certain item does not increase when it reaches nearly 18MB, indicating that it may not be a memory leak, but some memory is used by OpenSSL as cache. There is no online solution to this problem. SSL_CTX_remove_session(); SSL_CTX_remove_session(); SSL_CTX_remove_session(); SSL_CTX_remove_session(); Session reuse can improve SSL communication efficiency, but Nebula does not need it for now.

NebulaLogic takes NebulaInterface as an SSL server and NebulaLogic as an SSL client, completing the Nebula framework SSL server and client function test, a simple stress test. The Nebula framework’s SSL communications have been tested and are ready for production, and will certainly continue to be refined in future applications. Openssl is really hard to use, and no wonder it’s been teased so much. Perhaps openSSL will be replaced with other SSL libraries in a Nebula version soon.

End of the 5.

With the Nebula framework supported by SSL, the test was not too complicated, but the process was tortuous and took a long time. Nebula’s overview of DEVELOPING SSL communications using OpenSSL is shared here for developers ready to use OpenSSL. If you found this article useful to you, don’t forget to go to Nebula’s Github or code cloud and give it to Star. Thank you.

<br/>

References:

  • Secure connection based on OpenSSL implementation
  • SSL API documentation
  • Https protocol details
  • HTTPS is the trend? See Tencent experts through Epoll+OpenSSL in high concurrency pressure measurement robot support HTTPS
  • Introduction to OpenSSL Programming (complete example included)
  • Analysis of the SSL connection establishment process
  • SSL Socket communication details
  • HTTPS from Principle to Application (3) : SSL/TLS
  • SSL/TLS handshake optimization in detail
  • Non-blocking/asynchronous (epoll) OpenSSL
  • Two HTTPS client examples based on OpenSSL
  • An introduction to the general process of establishing SSL communication using the OpenSSL API
  • A preliminary study on OpenSSL programming 2 — Loading certificate files