Beginners 101 Guide To Always Encrypted

Monica Rathbun Consultant Denny Cherry & Associates Consulting PASS Mid-Atlantic Regional Mentor User Group Leader: Hampton Roads SQL Server User Group MRathbun@sqlespresso.com http://www.sqlespresso.com @SQLEspresso http://www.linkedin.com/in/sqlespresso

Presentation Rules Always ask Questions Interrupt Me This is a two way conversation, let’s learn from each other’s experiences.

Agenda The Basic Terminology Types of Encryption What is Always Encrypted (AE)? Key Storage How to Encrypt/Decrypt GOTCHAS vNext Enclaves

Encryption terminology Encrypt & Decrypt Column Encryption Key (CEK) Column Master Keys (CMK) Certificate Key Vaults A column encryption key is used to encrypt data in an encrypted column. A column master key is a key-protecting key that encrypts one or more column encryption keys. The Database Engine stores encryption configuration for each column in database metadata. Note, however, the Database Engine never stores or uses the keys of either type in plaintext. It only stores encrypted values of column encryption keys and the information about the location of column master keys, which are stored in external trusted key stores, such as Azure Key Vault, Windows Certificate Store on a client machine, or a hardware security module. Certificate = acts like an SSL certificate in the manner that it can encrypt/decrypt data. SSL Certificates are small data files that digitally bind a cryptographic key to an organization's details. When installed on a web server, it activates the padlock and the https protocol and allows secure connections from a web server to a browser. Asymmetric = one public key, one private key. Need both to unlock the data. Public key encrypts the data, private key decrypts the data Symmetric = one key to do both

Types of encryption Type Usage Things to Note Transparent Data Encryption (TDE) Database Level Data at Rest, Decrypted while in motion from Memory to Storage processor. Column Level Encryption Column Level DBA Can Get to Data, SQL Knows The Keys Dynamic Data Masking Not Really Encryption Always Encryption Encrypted Everywhere For Everyone Transparent Data Encryption (TDE) encrypts SQL Server, Azure SQL Database, and Azure SQL Data Warehouse data files, known as encrypting data at rest. You can take several precautions to help secure the database such as designing a secure system, encrypting confidential assets, and building a firewall around the database servers. However, in a scenario where the physical media (such as drives or backup tapes) are stolen, a malicious party can just restore or attach the database and browse the data. One solution is to encrypt the sensitive data in the database and protect the keys that are used to encrypt the data with a certificate. This prevents anyone without the keys from using the data, but this kind of protection must be planned in advance. Column Encryption – Not straight forward Create Cert, Create a symmetric key tell it what cert to use to decrypt . DBAs can get to data depending on how system is configured and how encryption is done. If encryption is done in sql then dbas can get to it. If done in an app then they cant.

MSDN defines Deterministic encryption as always generates the same encrypted value for any given plain text value. Which means that if you have a birthdate of 01/03/1958 it will always be encrypted with the same value each time such as ABCACBACB. This allows you to index it, use it in WHERE clauses, GROUP BY and JOINS. Randomized encryption per MSDN- uses a method that encrypts data in a less predictable manner. This makes Randomized encryption more secure, because using the example above each encrypted value of 01/03/1958 will be different. It could be ABCACBACB, BBBCCAA, or CCCAAABBB. All three encrypted values are subsequently decrypted to the same value. Since the encrypted value is random you cannot perform search operations etc. as you can with Deterministic. Available in Standard 2016 SP1 STANDARD Can be used to Block DBA’s from seeing the data CEK is stored outside of SQL Server No Code Changes Needed* Data Encrypted as it is SENT to application What Is AE ? CEK Column Encryption Key – you can have a different key per column or same keep for table Always Encrypted is a feature designed to protect sensitive data, such as credit card numbers or national identification numbers (for example, U.S. social security numbers), stored in Azure SQL Database or SQL Server databases. Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the Database Engine ( SQL Database or SQL Server). As a result, Always Encrypted provides a separation between those who own the data (and can view it) and those who manage the data (but should have no access). By ensuring on-premises database administrators, cloud database operators, or other high-privileged, but unauthorized users, cannot access the encrypted data, Always Encrypted enables customers to confidently store sensitive data outside of their direct control. This allows organizations to encrypt data at rest and in use for storage in Azure, to enable delegation of on-premises database administration to third parties, or to reduce security clearance requirements for their own DBA staff. No code changes needed out of box but for best performance code changes may be needed

TWO Types of AE Randomized Deterministic ABCACBACB WHERE clauses, GROUP BY and JOINS. Indexes Randomized ABCACBACB, BBBCCAA, or CCCAAABBB More Secure Non Searchable MSDN defines Deterministic encryption as always generates the same encrypted value for any given plain text value. Which means that if you have a birthdate of 01/03/1958 it will always be encrypted with the same value each time such as ABCACBACB. This allows you to index it, use it in WHERE clauses, GROUP BY and JOINS. Randomized encryption per MSDN- uses a method that encrypts data in a less predictable manner. This makes Randomized encryption more secure, because using the example above each encrypted value of 01/03/1958 will be different. It could be ABCACBACB, BBBCCAA, or CCCAAABBB. All three encrypted values are subsequently decrypted to the same value. Since the encrypted value is random you cannot perform search operations etc. as you can with Deterministic. In most cases, you will want to use deterministic encryption. The places where random encryption makes sense is where you have a low range of distinct values. An attacker might be able to determine what the encrypted value was by brute force attacking using a variety of parameters. Some examples of this data include birth date, blood type, or credit card verification numbers (CVV).

AE Gotchas Distributed Queries (linked servers) No Default Or Check Constraints No Partition Columns Columns Reference By Computed Columns No Transactional/Merge Replication Aggregations Columns with the IDENTITY property No Triggers Columns using one of the following datatypes: xml, timestamp/rowversion, image, ntext, text, sql_variant, hierarchyid, geography, geometry, alias, user defined-types. FILESTREAM columns Columns with the IDENTITY property Columns with ROWGUIDCOL property String (varchar, char, etc.) columns with non-bin2 collations Columns that are keys for nonclustered indices using a randomized encrypted column as a key column (deterministic encrypted columns are fine) Columns that are keys for clustered indices using a randomized encrypted column as a key column (deterministic encrypted columns are fine) Columns that are keys for fulltext indices containing encrypted columns both randomized and deterministic Columns referenced by computed columns (when the expression does unsupported operations for Always Encrypted) Sparse column set Columns that are referenced by statistics Columns using alias type Partitioning columns Columns with default constraints Columns referenced by unique constraints when using randomized encryption (deterministic encryption is supported) Primary key columns when using randomized encryption (deterministic encryption is supported) Referencing columns in foreign key constraints when using randomized encryption or when using deterministic encryption, if the referenced and referencing columns use different keys or algorithms Columns referenced by check constraints Columns in tables that use change data capture Primary key columns on tables that have change tracking Columns that are masked (using Dynamic Data Masking) Columns in Stretch Database tables. (Tables with columns encrypted with Always Encrypted can be enabled for Stretch.) Columns in external (PolyBase) tables (note: using external tables and tables with encrypted columns in the same query is supported) Table-valued parameters targeting encrypted columns are not supported.

STORING kEYS Local to Server Azure Subscription Reliable Internet WINDOWS CERTIFICATE STORE Local to Server https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/how-to-view-certificates-with-the-mmc-snap-in AZURE KEY Vault Azure Subscription Reliable Internet https://azure.microsoft.com/en-us/services/key-vault/ternet Connection One of the most complex parts of encryption is determining where to store these keys and who will have access to it.  You can store these keys on a client machine using a Windows Certificate store or in Azure Key store.’ Azure key vault enables SQL Server to protect the data encryption keys by using an asymmetric key stored outside of SQL Server in an external cryptographic provider. This model adds an additional layer of security and separates the management of keys and data. To ensure Always Encrypted is effective in preventing these types of attacks, your key management process must ensure the column master keys and column encryption keys, as well as credentials to a key store containing the column master keys, are never revealed to a potential attacker. Here are a few guidelines, you should follow: Never generate column master keys or column encryption keys on a computer hosting your database. Instead generate the keys on a separate computer, which is either dedicated for key management, or is a machine hosting applications that will need access to the keys anyway. This means that you should never run tools used to generate the keys on the computer hosting your database because if an attacker accesses a computer used to provision or maintain your Always Encrypted keys, the attacker can potentially get your keys, even if the keys only appear in the tool’s memory for a short time. To ensure your key management process does not inadvertently reveal column master keys or column encryption keys, it is critical to identify potential adversaries and security threats before defining and implementing a key management process. For example, if your goal is to ensure DBAs have no access to sensitive data, then a DBA cannot be responsible for generating the keys. A DBA, however, can manage key metadata in the database, as the metadata does not contain the plaintext keys. STORING kEYS

Windows certificate store On a computer that has the Windows operating system installed, the operating system stores a certificate locally on the computer in a storage location called the certificate store. A certificate store often has numerous certificates, possibly issued from a number of different certification authorities (CAs). If your column master key is a certificate stored in the local machine certificate store location, you need to export the certificate with the private key and import it to all machines that host applications that are expected to encrypt or decrypt data stored in encrypted columns, or tools for configuring Always Encrypted and for managing the Always Encrypted keys. Also, each user must be granted a read permission for the certificate stored in the local machine certificate store location to be able to use the certificate as a column master key. If your column master key is a certificate stored in the current user certificate store location, you need to export the certificate with the private key and import it to the current user certificate store location of all user accounts running applications that are expected to encrypt or decrypt data stored in encrypted columns, or tools for configuring Always Encrypted and managing Always Encrypted keys (on all machines containing those applications/tools). No permission configuration is required - after logging on to a machine, a user can access all certificates in their current user certificate store location. Two certificate locations: local machine or current user Key store exists on each machine hosting your application

AZURE KEY VAULT Extra Security Level Role Separation Key Logs to Audit Key Usage The SQL Server Connector for Microsoft Azure Key Vault enables SQL Server encryption to use the Azure Key Vault service as an Extensible Key Management (EKM) provider to protect SQL Server encryption keys. The SQL Server Connector serves as a bridge between SQL Server and Azure Key Vault, so SQL Server can leverage the scalability, high performance, and highly availability of the Azure Key Vault service. The following image represents how the key hierarchy works in the EKM provider architecture with Azure Key Vault and SQL Server Connector. Azure Key Vault can be used with SQL Server installations on Microsoft Azure Virtual Machines and for on-premises servers. The key vault service also provides the option to use tightly controlled and monitored Hardware Security Modules (HSMs) for a higher level of protection for asymmetric encryption keys. For more information about the key vault, see Azure Key Vault. Azure Key Vault helps safeguard cryptographic keys and secrets used by cloud applications and services. By using Key Vault, you can encrypt keys and secrets (such as authentication keys, storage account keys, data encryption keys, .PFX files, and passwords) using keys protected by hardware security modules (HSMs). For added assurance, you can import or generate keys in HSMs. If you choose to do this, Microsoft processes your keys in FIPS 140-2 Level 2 validated HSMs (hardware and firmware). Key Vault streamlines the key management process and enables you to maintain control of keys that access and encrypt your data. Developers can create keys for development and testing in minutes, and then seamlessly migrate them to production keys. Security administrators can grant (and revoke) permission to keys, as needed. AZURE KEY VAULT

Encryption DEMO Show how to encrypt Show to create cert and where it is stored in MMC – add on snapin

Decryption in SSMS SSMS uses .NET 4.6 so you can pass in the necessary encryption options. SSMS uses the connection string to access the Master Key and return the data in its decrypted format. Windows certificate store.  .NET 4.6 ships with the SqlColumnEncryptionCertificateStoreProvider class, which contains all the logic necessary for accessing the certificate store of the machine where that code is running.  Providers that are included in .NET are pre-registered with the ADO.NET driver, saving you the effort of doing so yourself. One thing that remains the same between the Windows certificate store and Azure Key Vault providers is that your connection string still needs to contain “Column Encryption Setting = Enabled” in order for your client application to know to try to transparently handle encryption. ** note the only way I can do this with Azure Key vault is with the correct permissions that’s how you lock the DBA out.

Decryption in Application Database Permissions VIEW ANY COLUMN MASTER KEY DEFINITION   VIEW ANY COLUMN ENCRYPTION KEY DEFINITION  These permissions are required to access the metadata about Always Encrypted keys in the database. string connectionString = "Data Source=server63; Initial Catalog=Clinic; Integrated Security=true; Column Encryption Setting=enabled"; SqlConnection connection = new SqlConnection(connectionString);

Decryption in Application Database Permissions VIEW ANY COLUMN MASTER KEY DEFINITION   VIEW ANY COLUMN ENCRYPTION KEY DEFINITION  These permissions are required to access the metadata about Always Encrypted keys in the database. string connectionString = "Data Source=server63; Initial Catalog=Clinic; Integrated Security=true; Column Encryption Setting=enabled"; SqlConnection connection = new SqlConnection(connectionString);

Decryption DEMO Demo seeing the data and remind about key vault permissions

VNEXT…Enclaves (In Preview) An enclave is a protected region of memory that acts as a trusted execution environment. An enclave appears as a black box to the containing process and to other processes running on the machine. There is no way to view the data or the code inside the enclave from the outside, even with a debugger. An enclave is a protected region of memory that acts as a trusted execution environment. An enclave appears as a black box to the containing process and to other processes running on the machine. There is no way to view the data or the code inside the enclave from the outside, even with a debugger. VNEXT…Enclaves (In Preview)

AE Using Enclaves (In Preview) Pattern Matching Range Comparisons Sorting Rich Computations Initial Data Encryption Rotating a Column Encryption Key Changing a Data Type of an Encrypted Column In-Place Encryptions While the new architecture will enable us to support a broad range of computations on encrypted columns inside the SQL Server Engine, we initially target the most commonly requested operations, such as pattern matching (LIKE), range queries, sorting (ORDER BY), as well as a broad set of data type conversions Enabling cryptographic operations locally, eliminates the need to move the data out of the database for initial encryption or for other encryption-related schema changes. This greatly improves performance and reliability of such operations and means the operations can be triggered via ALTER TABLE ALTER COLUMN T-SQL statements and do not require special client-side tools. Here is an example of a statement that triggers a rotation of a column encryption key:

Questions?

MSDN https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/always-encrypted-database-engine?view=sql-server-2017 Secure Enclaves https://blogs.msdn.microsoft.com/sqlsecurity/2017/10/05/enabling-confidential-computing-with-always-encrypted-using-enclaves-early-access-preview/ TDE https://docs.microsoft.com/en-us/sql/relational-databases/security/encryption/transparent-data-encryption?view=sql-server-2017 Aaron Bertrand Blog https://blogs.sentryone.com/aaronbertrand/t-sql-tuesday-69-always-encrypted-limitations/ Resources