openUTM uses either a combination of the algorithms RSA (named after the authors Rivest, Shamir and Adleman) and AES-CBC (Advanced Encryption Standard Cipher Block Chaining Mode), or a combination of Ephemeral Elliptic Curve Diffie-Hellman (ECDHE) and AES-GCM (Advanced Encryption Standard Gallois Counter Mode). The second combination is more modern and offers advanced security compared to the first combination, but is currently only supported by openUTM on Unix, Linux and Windows systems.
RSA-AES-CBC cipher suite
With the combination RSA-AES-CBC, before transmission the AES key is encrypted with the public RSA key of the UTM application. In order to do so openUTM generates an RSA key pair consisting of a public and a private secret key.
For a UTM application RSA keys with a length of 1,024 and/or 2,048 bits can be generated.
BSI recommends using RSA keys with a length of at least 2,000 bits.
- The public RSA key is transferred from the UTM application to the client when the connection is established. To prevent man-in-the-middle (MiM) attacks on the communication, a user should also read out the public RSA key of the application via administration, transfer it separately to the client and enter it into the client configuration.
- For each new connection, the client generates a new 128-bit AES key, encrypts it with the public RSA key of the server, and transmits it to the UTM application. The AES key is connection-specific, i.e. a separate key is generated for each connection and this key is only used for this connection.
- The UTM application decrypts the AES key using its private RSA key.
User data and passwords are encrypted on a connection with the symmetric AES key, i.e. client and UTM application use the same AES key to encrypt and decrypt messages.
ECDHE-RSA-AES-GCM cipher suite (only for Unix, Linux and Windows systems)
The combination ECDHE-RSA-AES-GCM uses the elliptic curve based Diffie-Hellman method to generate an AES session key. Each side generates a Diffie-Hellman key pair, transmits the public part of its key pair to the partner and generates the common AES session key with its private key and the public key of the partner. This means that in this procedure the AES session key is not transferred on the data connection.
The server also signs its public Diffie-Hellman key with the private RSA key of the UTM application. In this way, the client can verify that the Diffie-Hellman public key sent from the server really belongs to the UTM application. Again, as described above, to defend against man-in-the-middle attacks, the public RSA key should be made known to the client separately.
The Ephemeral Diffie-Hellman method offers the user Perfect Forward Secrecy; this means that even recorded data cannot be decrypted later if the long-term key (RSA key) should later be cracked.
The AES-GCM algorithm is used to encrypt user data. One of the advantages of this method over AES-CBC is that it supports Authenticated Encryption with Associated Data (AEAD), in which the encrypted user message and the other protocol parts of the message are protected against changes by a Message Authentication Code (MAC).
Passwords are encrypted with AES-CBC as described above.