1. Always Encrypted in the Real World
Daniel de Sousa
Always Encrypted in the Real World

3. About Me Daniel de Sousa
Business Intelligence large pizza chain
Currently working on data compliance project between multi countries and continents
Been working with data since 1994(dBase III Plus, all SQL servers)
Currently using SQL 2016 SP1 
Currently living in Brisbane, Australia

4. Agenda Scenario Challenges Solution Technology used Limitations
Demo/Feature use

5. Scenario

6. Scenario Large Global Pizza Chain Headquarters: Franchises: Australia
Belgium
Germany
France
Netherlands
Japan
New Zealand

7. Scenario Process: Stores provide daily data to HQ
Data is received, grouped and processed (in AU)
Data is loaded into datawarehouse (in AU)
Franchisees can access data(Cube/Report/Dashboard)
Global Reports are generated

8. Challenges

9. Challenges
Although they have same system, it is used differently between countries
Data coding(varchar x nvarchar)
Reports built based on AU data
Any delay/failure in AU process, delays more data access anywhere
Europen data(pii) cannot leave EU without encryption

10. Solution

11. Solution Datawarehouse stays in AU Azure VM on regions SQL 2016 SP1
Azure functions
Azure KeyVault
Always Encrypted
Encryption

12. Technology used

13. Technology used Azure VM on regions 4x VMs in Western Europe
SQL 2016 SP1
SSIS 2016
SSRS 2016
SSAS 2016
3x VMs in Asia

14. Technology used Azure Functions
Functions were created to trigger the data from the store at the end of the day as well as to consume the data received
Staged the received data(more for control)

15. Technology used Azure KeyVault
The security guys create a secret for each “type” of application
Avoid inter-dependence and total failure in case of small failures
Easier to rotate in case a secret is exposed
Easier to rotate on a time basis

16. Technology used Always Encrypted SQL Server 2016 onwards
All editions(from SP1)
Client Side Encryption – ADO .Net
Requires .Net 4.6 driver

17. Technology used Always Encrypted
The Certificate protects the key and is needed for encryption/decryption
The Certificate is needed on the client, not the server
Certificates can be created by
SQL Server
Makecert
New-SelfSignedCertificate
Certificate Store – Current User
Certificate Store – Local computer
Azure Key Vault
Key Store Provider (CNG)
1) indicates the Current User certificate store location in the Windows Certificate Store, which is your personal store.
2) indicates the Local computer certificate store location in the Windows Certificate Store.
3) you will need to sign in to Azure (click Sign in). Once you sign in, you will be able to pick one of your Azure subscriptions and a key vault.
4) indicates a key store that is accessible via a key store provider (KSP) that implements the Cryptography Next Generation (CNG) API. Typically, this type of a store is a hardware security module (HSM). After you select this option, you will need to pick a KSP. Microsoft Software Key Store Provider is selected by default. If you want to use a column master key stored in an HSM, select a KSP for your device (it must be installed and configured on the computer before you open the dialog).
5) a key store that is accessible via a cryptographic service provider (CSP) that implements the Cryptography API (CAPI). Typically, such a store is a hardware security module (HSM). After you select this option, you will need to pick a CSP. If you want to use a column master key stored in an HSM, select a CSP for your device (it must be installed and configured on the computer before you open the dialog).

18. Technology used Always Encrypted Column Master Key(CMK)
Column Encryption Key(CEK)
Encrypt Column (AEAD_AES_256_CBC_HMAC_SHA_256)
Drivers(.NET or ODBC 13.1)
Connection String
CMK
Protects column encryption keys.Must be stored in a trusted key store.Stored in the database in system catalog views.
CEK
Encrypt sensitive data stored in database columns.Column can be encrypted using a single column encryption key.Encrypted values of column encryption keys are stored in the database in system catalog views.Backup column encryption keys in a secure/trusted location
CMK protects the CEK and it’s stored in trusted key stores such as Windows Certificate Store(WCS), Azure keyVault, Hardware Security Modules or even custom key store if necessary.
IMPORTANT: In a production environment, the certificate management must be done with separate, secured user accounts and, preferably, on separate servers.
CEK protects the encrypted columns and it’s stored encrypted in the database.
Data in the db is always stored encrypted using AEAD_AES_256_CBC_HMAC_SHA_256.
All decryption is done by client application which requires AE-enabled client driver.
Deterministic: same encrypted value for any unencrypted value which allows creation of indexes, point-lookup searches, equality joins and grouping. Only one that can be indexed.
Randomized: it generates a new value for each encryption
CONNECTION STRING:
Driver={ODBC Driver 13 for SQL Server};server=localhost;trusted_connection=Yes;database=DIME;columnencryption=Enabled;keystoreauthentication=KeyVaultClientSecret;keystoreprincipalid=XXXxxxXXXxxxXXXxxx;keystoresecret=fxfxfxfffkhskdjfskdfgskdfhskdjvsdf;sodfhskudgfksdj

19. Technology used Always Encrypted SSMS(Advanced SSMS
SSMS 17 (config/parameterization)
Column Encryption Setting = Enabled
SSMS 16.3
To enable/disable Parameterization for Always Encrypted for the current Query Editor window: +
Select Query from the main menu.
Select Query Options….
Navigate to Execution > Advanced.
Select or unselect Enable Parameterization for Always Encrypted.
Click OK. +
To enable/disable Parameterization for Always Encrypted for future Query Editor windows: +
Select Tools from the main menu.
Select Options….
Navigate to Query Execution > SQL Server > Advanced.
SSMS 17
This allows the underlying .NET Framework Data Provider for SQL Server to detect data targeting encrypted columns, and to encrypt such data before sending it to the database
Without parameterization, the .NET Framework Data Provider passes each statement, you author in the Query Editor, as a non-parameterized query. If the query contains literals or Transact-SQL variables that target encrypted columns, the .NET Framework Data Provider for SQL Server will not be able to detect and encrypt them, before sending the query to the

20. Technology used

21. Technology used

22. Technology Used Encryption Text Data is varbinary
Create a number to represent(Encrypt)
Processed in Australia
Flow back to Region
Map number back(Decrypt)

23. Limitations

24. Limitations Deterministic Encryption requires _BIN2 collation.
A CEK can have two encrypted values (for key rotation)
Queries can only perform operations on deterministic encryption
Only the equals (=) operation is allowed in queries. (No >, <, <>, like, etc.)
Queries must pass values as parameters, not literals.
Limited data types
Key columns in indexes only allow deterministic encryption
NO CDC
NO FOR XML or FOR JSON PATH
During Encryption process, there can be no inserts or updates as it may result in data corruption
And More can be checked at Microsoft Docs(reference)

25. Demo A look at the feature
Create CMK
Create CEK
Create table
Encrypt column
Create table with Encrypted column
Insert data
Select data without setting
Select data with setting
Select data with CLAUSE

26. References Always Encrypted (Database Engine)
Always Encrypted
Column Encryption Metadata

27. Questions?

28. THANK YOU