Explaining how TLS/SSL and certificates work
In this post I will explain the basics of TLS, because there is a lot of misinformation on the internet and youtube especially. Then I will analyze a TLS connection with wireshark
Transport Layer Security is the successor of SSL and provides confidentiality, data integrity, and (mutual) authentication. TLS runs on top of TCP, and sits between the transport layer and the application layer. TLS is not a single protocol but a set of protocols.
As we can see in the following figure, there are protocols that run during the handshake phase:
- handshake protocol : authenticate entities, negotiate cipher suites, key exchange
- change cipher spec protocol: change cipher for use in connection
- alert protocol : alert peer entity of status, warning, errors
- record layer protocol : provides confidentiality and integrity. Its functions are breaking all outgoing messages into blocks, compressing them, adding a MAC and encrypting them. On the incoming side, it does the opposite things. Each block is decrypted, decompressed, MAC verified and reassembled. A total of four keys are used (MAC + encryption in one direction, MAC + Encryption in the other direction).
When using https, all http data are transfered in the record layer.
Confidentiality is achieved through encryption. There are two main categories of encryption algorigthms.
- Symmetric algorithms like AES, use a single key for encryption and decryption
- They are always used for the encryption of data, because they are more cpu efficient.
- Symmetric keys are known also as session keys, because they are used for a single session and then discarded. (actually if the session takes long, they can be regenerated)
- Asymmetric algorithms like RSA, DH, use two separate keys.
- Whatever is encrypted with one key, can be decrypted with the other.
- The same key cannot be used for encryption and decryption.
- Public Key Infrastructure uses asymmetric algorithms. One key is distributed and is called public, and the other is kept confidential and called private. Here is the first misconception.
Some people think that the public key can be used only for encyption and the private only for decryption. Although this is the common case, the truth is you can do both: encrypt with private and decypt with public, OR, encrypt with public and decrypt with private. Of course if you encrypt with the private key, then anyone can decrypt it, so the usual case is that the public key is used for encryption, while the secret key is used for signing (more on this later).
I have read many times that the client generates a session key, encrypts it with the public key from the server and sends it to the server. The server decrypts it with its private key, and then uses this decrypted key for encryption and decryption of data. The truth is that the session keys are never exchanged, but they are generated independently on each side, based on other data they have exchanged.
Session keys are computed on each side of the connection. The components used to generate the session keys are passed between the client and the server in one of two ways.
- key exchange (RSA,DSA): the client generates a pre-master secret and sends it to the server, encrypted with the server’s public key. Both sides use this key and parameters exchanged during the handshake process to generate the master secret. The master secret is used in a Pseudo Random Function to generate the actual session keys.
- key generation (DH, ECDH): The client and server independently generate a pre-master secret, after an initial exchange of components, that are required for the process, all of which can be public and therefore do not require encryption. Each side adds components that must remain private. The components are not transmitted across the network, but remain on each side. The pre-master key is used with other values to generate the encryption and MAC keys (one for each direction - total 4)
Advantage of DH over RSA
In DH, if an attacker intercepts the traffic during the handshake, it does not have enough information to compute the master secret and thus derive the keys, even if it has the server’s private key.
In RSA, if an attacker gets hold of server’s private key, it can derive the symmetric keys and decrypt the exchanged data.
A MAC is used to make sure that nobody has tempered with the data in transit and also provides authentication. This is achieved, because MAC uses a private key as the seed to the hash function it uses. This assures the receiver, that not only the message hasn’t been modified, but also who sent it, is the one you expect it from. HMAC is a hash based message authentication code.
Certificates are used for authentication and not for encryption. Usually only the server is authenticated and not the client. However TLS supports also client authentication. Certificates allows us to trust sites, that a third trusted party has said that they are who they claim to be.
Usually a company who wants a certificate, creates a certificate sign request (the csr has domain name, name of company, email, location etc) and sends it to a certificate authority, like verisign.
The CA after it checks the authenticity of the company, creates a certificate and signs it.
What is signing?
Signing your certificate, means that your data is hashed and encrypted with the private key of the CA. The other side decrypts the message with the public key and compares the decrypted hash with the hash of the received message. If they agree, you can trust the certificate.
The certificate that you ordered from the CA is installed on the web server.
When the browser visits the server, the server sends the certificate, which was signed by the CA (along with certificates from ICA, more on this later).
The browser has preinstalled all the root certificates in a trusted web store. Each certificate includes the public key, so it can decrypt the signature, which was signed by the CA. If the decrypted hash matches the hash of the received message, it is sucessful. Some additional checks are performed, like if the domain name matches the one on the certificate, if the certificate hasn’t expired or revoked. If all are successful the browser trusts the site.
Intermediate and root CAs, certificate chain
There are 2 types of Certificate Authorities: root CAs and intermediate CAs (ICA).
Usually your certificate will be signed by an ICA, and not from the root CA. This is done for security reasons. So your certificate will be signed by ICA, but your browser may not have the ICA’s certificate in its store.
What happens, is that your server will send along with its certificate, the certicate of the ICA, which is signed by the root CA. (actually you can have a lot of ICAs, but for the sake of simplicity we assume one). Because the browser has the root CA, and the ICA is signed by the CA, it will trust the certificate signed by the ICA.
browser trusts CA, CA trusts ICA, ICA trusts your webserver => browser trusts your webserver
The list of SSL certificates, from the root certificate to the end-user certificate, represents the SSL certicate chain
in the above picture we see that the site www.google.com has a certificate issued by an ICA which is Google Internet Authority G3, which has a certificate issued by Google Trust Services, which is a Root CA
Wireshark Analysis of a TLS handshake process
if you want to download the pcap file click here.
After the TCP handshake, SYN,SYN-ACk,ACK, we have the TLS handshake
SSL handshake Protocol
- allows client and server to authenticate each other
- negotiate encryption and MAC algorightms, exchange keys
in summary the steps are:
- client sends client_hello that contains the supported cipher suites
- server sends server_hello that contains the cipher selected
- server sends its certificate singned by the CA and the ICAs
- server sends key exchange message which contains the parameters needed for the asymmetric algorigthm, which is usually ECDH (elliptic curve Diffie Hellman)
- server sends certificate request (optional for client authentication)
- server sends hello_done
- client sends certificate (if requested)
- client sends key exchange message which contains the parameters needed for the asymmetric algorithm
- client sends certificate_verify
- client sends change_cipher_spec message that requests the server to use the new keys
- client sends finished message
- server sends change_cipher_spec that requests the client to use the new keys
- server sends finshed message
a lot of the above messages can be combined in one packet as we shall see
The client send a TLS hello packet with all the cipher suites that it supports. The message contains also a random byte string that is used in later computations.
The server sends a TLS hello packet with the cipher suite that it selected (ECDHE, ellicptic curve diffie Hellman Ephemeral for key exchange, RSA for signing, AES 256 for symmetric encryption, GCM is the mode of operation of AES, SHA384 for MAC), the session ID and another random number. (If the server requires client authentication, it will also send a client certificate request.)
The server sends its public certificate, along with all the other ICA certificates in the certificate chain, a certificate status, a server key exchange, which includes the parameters used for the DH algorigthm, and server hello done. (click on the image to enlarge)
The certificate status message validates whether the server’s X.509 digital certificate is revoked or not, it is ascertained by contacting a designated OCSP (Online Certificate Status Protocol) server.
this was requested in the client hello message
Client Key Exchange, Change Cipher Spec, Finished
The client sends a client key exchange message, which includes the parameters for DH key exchange, a change cipher spec and encrypted handshake message. The change cipher spec protocol is used to inform the server that all the subsequent messages will be encrypted with the negotiated algorithms and keys. What wireshark shows as encrypted handshake message is the finished message which is a hash of all the messages previously exchanged along with a label (“client finished”), and indicates that the the tls negotiation is finished for the client. (click on the image to enlarge)
TLS session ticket, Change Cipher Spec, Finished
The server sends a TLS session ticket, a change cipher spec and an encrypted handshake message. The TLS session ticket identifies the session. The TLS session is an association between the client and the server. Multiple connections can be associated with one session. Security parameters for session can be shared among connections. Session state is stored after the handshake.When a new connection is established, the client and the server agree to reuse the previous keys, so they only do an abbreviated handshake instead of a full handshake. Because the generation of the keys is very computationally intensive, using the same tls security parameters increases the performace by about 20%.
The change cipher spec protocol is used to inform the client that all the subsequent messages will be encrypted with the negotiated algorithms and keys. What wireshark shows as encrypted handshake message is the finished message which is a hash of all the messages previously exchanged along with a label (“server finished”), and indicates that the the tls negotiation is finished for the client. (click on the image to enlarge)
The client and server can now exchange application data using the symmetric encryption and session key.