TLS/SSL Protocol

Peter Kacherginsky
5 min readApr 12, 2009

Transport Layer Security (TLS) and Secure Sockets Layer (SSL) are two closely related protocols designed to protect confidentiality and integrity of data in transit between two hosts.

TLS/SSL can add a secure layer to a number of already existing application layer protocols (e.g. HTTP, LDAP, FTP, SMTP, and others). It can also be used to create a VPN solution (e.g. OpenVPN).

History

SSL protocol was developed by Netscape in 1994. In 1996 Netscape released SSL 3.0 which is would be the last version. IETF continued the development of the protocol under a new name — Transport Layer Security (TLS). In 1999 IETF released TLS 1.0 as a standard in RFC2246.

Architecture

The TLS/SSL Protocol is composed of multiple layers. The lowest level, layered on top of some reliable transport protocol (e.g. TCP), is the Record Protocol. It is used for encapsulation of several higher level protocols:

  • Handshake Protocol — allows the server and client to authenticate each other, to negotiate an encryption algorithm, and exchange cryptographic keys.
  • Alert Protocol — alert messages convey the severity and a description of the alert (errors, closure, handshake failures, etc.). Alert messages with a level of fatal result in the immediate termination of the connection. Alert messages are encrypted and compressed, as specified by the current connection state.
  • Application Protocol — protocol that normally layers directly on top of the transport layer (e.g., TCP/IP). Examples include HTTP, TELNET, FTP, and SMTP.

Handhake Protocol

In order to establish a secure session between a client and a server, TLS protocol performs several authentication and shared key creation steps known as the Handshake. During the handshake, the following messages may be exchanged:

      Client                                               Server

ClientHello -------->
ServerHello
Certificate*
ServerKeyExchange*
CertificateRequest*
<-------- ServerHelloDone
Certificate*
ClientKeyExchange
CertificateVerify*
[ChangeCipherSpec]
Finished -------->
[ChangeCipherSpec]
<-------- Finished
Application Data <-------> Application Data

* - optional

Below is a sample Handshake exchange between a client and a server:

Client Hello

Client Hello — the client transmits a hello message in order to establish security enhancement capabilities between client and server. Client Hello message contains client’s protocol version, session id (used to resume previously cached handshake), a list of supported ciphers suites, a list of compression methods, and random data.

Cipher Suites can be interpreted as the following suite of cipher algorithms:

[Key Exchange]_[Authentication]_WITH_[Data Transfer Cipher]_[MAC Digest]

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 157
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 153
Version: TLS 1.0 (0x0301)
Random
gmt_unix_time: Apr 2, 2009 22:38:27.000000000
random_bytes: 32B23A0347D9ECB6DF266DDC9F0FA5C16F1246EFF18E6603...
Session ID Length: 0
Cipher Suites Length: 72
Cipher Suites (36 suites)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
...
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 40
Extension: server_name
Extension: elliptic_curves
Extension: ec_point_formats
Extension: SessionTicket TLS

Server Hello

Server Hello — the server transmits its own hello message in response to client’s hello. It contains server’s choice of the protocol version, session id (attempts to match session id received from the client), the chosen cipher suite, the chosen compression method, and random data.

In the example below, TLS_RSA_WITH_AES_256_CBC_SHA cipher suite selection can be interpreted as follows:

  • Key Exchange Algorithm — TLS
  • Authentication Algorithm — RSA
  • Cipher for Data Transfer — AES 256 CBC Block Cipher
  • MAC Digest Algorithm — SHA

Below is a sample Server Hello packet snippet:

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Server Hello
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 74
Handshake Protocol: Server Hello
Handshake Type: Server Hello (2)
Length: 70
Version: SSL 3.0 (0x0300)
Random
gmt_unix_time: Apr 2, 2009 22:38:27.000000000
random_bytes: 7987623FE44C3EB70533EDA934E8ED597FBD65F80467D1B2...
Session ID Length: 32
Session ID: 937AE95B176855A3DCAF555EB1D54F96C537D1ADC3D8B3E1...
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Compression Method: null (0)

Certificate

Certificate — the server sends its certificate which includes server’s public key. The client may use this information to verify authenticity of the server.

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Certificate
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 553
Handshake Protocol: Certificate
Handshake Type: Certificate (11)
Length: 549
Certificates Length: 546
Certificates (546 bytes)
Certificate Length: 543
Certificate (id-at-commonName=localhost)

Server Hello Done

Server Hello Done — the server completes it’s hello messages. Protocol, cipher, and compression negotiations are complete.

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Certificate
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 553
Handshake Protocol: Certificate
Handshake Type: Certificate (11)
Length: 549
Certificates Length: 546
Certificates (546 bytes)
Certificate Length: 543
Certificate (id-at-commonName=localhost)

Client Key Exchange

Client Key Exchange — with this message, the premaster key is set through direct transmission of the RSA-encrypted secret which will allow each side to agree upon the same premaster secret. Other key exchange algorithms are supported. (Diffie-Hellman key exchange is deprecated).

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Client Key Exchange
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 132
Handshake Protocol: Client Key Exchange
Handshake Type: Client Key Exchange (16)
Length: 128

Computing the master secret

Once the client generates and sends out the premaster key, both client and server will compute the master key using a PRF (Pseudo Random Function). PRF takes a key (premaster key sent by the client), a label (master key), and a seed (client random value + server random value). PRF will generate a 48 byte master key by splitting the premaster key in half and XORing those halves with MD5 and SHA-1 hashes of respective halves.

PRF(secret, label, seed) = P_MD5(S1, label + seed) XOR
P_SHA-1(S2, label + seed);

Client Change Cipher Spec

Change Cipher Spec — the client notifies the server, that all further communication will be encrypted

Secure Socket Layer
SSLv3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
Content Type: Change Cipher Spec (20)
Version: SSL 3.0 (0x0300)
Length: 1
Change Cipher Spec Message

Client Encrypted Finished

Encrypted Finished — this message is sent by the client to verify that the key exchange and authentication processes were successful. The finished message is protected with the just-negotiated algorithms, keys, and secrets. Once the other party receives and validates this message, it will be able to send application data over the connection.

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Encrypted Handshake Message
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 64
Handshake Protocol: Encrypted Handshake Message

Server Change Cipher Spec

Change Cipher Spec — upon the receipt of a valid Finished message from the client, the server will also announce that it is switching to encrypted communication with its own Change Cipher Spec announcement.

Secure Socket Layer
SSLv3 Record Layer: Change Cipher Spec Protocol: Change Cipher Spec
Content Type: Change Cipher Spec (20)
Version: SSL 3.0 (0x0300)
Length: 1
Change Cipher Spec Message

Server Encrypted Finished

Encrypted Finished — at last the server will issue its own encrypted Finished message to make sure the client and the server can understand each other. This concludes the handshake.

Secure Socket Layer
SSLv3 Record Layer: Handshake Protocol: Encrypted Handshake Message
Content Type: Handshake (22)
Version: SSL 3.0 (0x0300)
Length: 64
Handshake Protocol: Encrypted Handshake Message

Application Protocol

Once both the client and the server have completed the handshake, they will be able to use an application protocol (e.g. HTTP) over a secure session.

Secure Socket Layer
SSLv3 Record Layer: Application Data Protocol: http
Content Type: Application Data (23)
Version: SSL 3.0 (0x0300)
Length: 400
Encrypted Application Data: D07B8A874B6E56A3E40144D3AABEA7454D955E9EF085535F...

External Links

--

--

Peter Kacherginsky

Blockchain Security, Malware Analysis, Incident Response, Pentesting, BlockThreat.net