SQL Server 2017 Linux and Beyond Andrew Loree

Got a question? Andrew Loree andyloree.com andy@andyloree.com @LowOnDiskSpace

Are you passionate about helping customers unleash the value of their data? Building mission critical solutions? Using hybrid database platforms that have everything built in – from in-memory performance and advanced security to in-database analytics? Joining a team of passionate people who share your curiosity and dedication to discovering the value of data? Come join our team of consultants and Premier Field Engineers at Microsoft Services. Send us your resume, we want to know more about you: MSServicesSQLjobs@microsoft.com.   To register for more Microsoft information: https://aka.ms/msservicessqlsaturdays If you’re interested in other positions with Microsoft, check out our current open jobs at Microsoft.com/ServicesCareers

Goals To understand: What is SQL Server 2017 Install and configure on Linux Running on Docker What else is new? What is next?

https://aka.ms/eapsignup What is SQL Server 2017 Next major release of SQL Server Currently in Customer Technology Preview (CTP 2.1) Monthly releases of new features and fixes Release notes CTP 1.0 (Nov 2016),…,CTP 2.0 (Apr 2017), CTP 2.1 (May 2017) RC1 (July 2017), currently RC2(Aug 2017) Lots of new features (beyond just SQL Server on Linux) Production support with Early Adoption Program (EAP) https://aka.ms/eapsignup Describe the purpose, link to source info, release notes, … http://bit.ly/sql-vnext-whats-new

Where can I run SQL Server 2017? Windows 64-bit (ISO and CAB) Linux (supported Linux distributions) Red Hat Enterprise (RHEL) 7.3 Ubuntu Linux 16.04 SUSE v12 SP2 Azure VM template (RHEL 7.3) Docker on Windows Image on Linux Image What about macOS or XYZ Distro? Not planned for support but likely to work or use Docker Windows 64-bit (ISO and CAB) - http://bit.ly/sql-vnext-ctp Linux (supported OS packages) Red Hat Enterprise (RHEL) 7.2 - http://bit.ly/sql-vnext-rhel Ubuntu Linux 16.04 - http://bit.ly/sql-vnext-ubuntu SUSE v12 SP2 - http://bit.ly/sql-vnext-suse Azure VM template (RHEL 7.3) - http://bit.ly/sql-vnext-azure Docker on Windows Container - http://bit.ly/sql-vnext-win-docker on Linux Container - http://bit.ly/sql-vnext-linux-docker macOS – use Docker

How does SQL Server run on Linux? SQL Server Operating System (SOS/SQL OS) Internal abstraction layer for tasks, schedules, memory,… Drawbridge project MSFT Research late 2011 Application sandboxing – isolating, picoprocesses, library OS,… SQL Platform Abstraction Layer (SQL PAL) Started with SQL 2005 design Reference https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/ https://www.microsoft.com/en-us/research/project/drawbridge/

How does SQL Server run on Linux? Started with SQL 2005 design Reference https://blogs.technet.microsoft.com/dataplatforminsider/2016/12/16/sql-server-on-linux-how-introduction/ https://www.microsoft.com/en-us/research/project/drawbridge/ Outcome: Effectively the same code for Windows runs on Linux

SQL Server on Linux Minimum system requirements: 4 GB memory (tested up to 1TB to date) 2 GB disk space minimum Supported file systems: ext4 or xfs https://wiki.mikejung.biz/Ubuntu_Performance_Tuning

SQL Server on Linux - Demo Install & config mssql-server command-line tools Sql server agent Restore a database

SQL Server on Linux - Administration Configuration – Port, SA password, default paths, traceflags /opt/mssql/bin/mssql-conf list /opt/mssql/bin/mssql-conf set network.tcpport 1433 Service control – systemd service manager systemctl enable mssql-server systemctl disable mssql-server systemctl start mssql-server systemctl stop mssql-server systemctl status mssql-server Logging Default path for errorlog, XE - /var/opt/mssql/log syslog by default for SQLPAL, so make sure to monitor

SQL Server on Linux - Administration Connecting – via ssh protocol – putty, ssh,... Command pipeline – similar to PowerShell’s | Admin commands su/sudo – “run as” admin File nano/vi/emacs,… - console text editing and more grep/sed/ack – text file patterns, regex, stream & transform ls/find/mkdir/rmdir/touch – navigate and modify files & folders Process and performance related top/htop - displays process and resource usage ps/kill – active processes, signal or terminate a process https://www.mssqltips.com/sql-server-tip-category/226/sql-server-on-linux/

SQL Server on Linux - Administration Network ifconfig – view, config & control interfaces traceroute & ping Storage parted – partition/formatting mkfs – format file system on a partition mount/unmount – attach/detach into the root file system df – mounted file system free space & usage du – used space for given directory chown/chmod/setfacl – file permissions/ACL’s lvcreate/lgcreate – create lvm disk, groups * General Linux admin support links in the slide notes https://www.mssqltips.com/sql-server-tip-category/226/sql-server-on-linux/

SQL Server on Linux - Security AD Authentication on Linux Using realmd open source package and Kerberos Single-signon, supports AD groups, no local account auth Connection encryption TLS 1.2, 1.1 & 1.0 on both Linux and Windows Transparent Data Encryption (TDE) Yep https://www.mssqltips.com/sql-server-tip-category/226/sql-server-on-linux/

SQL Server on Linux – High Availability Shared-disk clustering using Pacemaker Pacemaker != WSFC Availability Groups All features supported Read-only routing Auto-seeding Distributed/subnets Failover options http://clusterlabs.org/

SQL Server on Linux - Performance Hardware/virtualization and OS vendor recommendations Storage configuration & separation rules still apply Tempdb, data, log, … Kernel tunables (sysctl): vm.swappiness = 10? vm.dirty_reads fs.file-max … Check out TPC-H support documents for SQL on Linux submission SQL configuration practices: max server memory, MAXDOP, trace flags,… ALWAYS BASELINE, CHANGE & TEST AGAIN! https://www.mssqltips.com/sql-server-tip-category/226/sql-server-on-linux/ https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-tunables.html http://www.tpc.org/tpch/results/tpch_perf_results.asp?resulttype=all

SQL Server on Linux - Monitoring Grafana – open-source analytics & monitoring How SQLCAT Customer Lab is monitoring SQL on Linux https://github.com/Microsoft/dbfs

SQL Server on Linux - Monitoring Most 3rd party tools will continue to work as they do with Windows hosts – DMV’s work as always DBFS open-source FUSE mount of SQL DMV’s DMV access, the linux sysadmin way https://github.com/Microsoft/dbfs

Machine vs OS Virtualization (containers) What is Docker? Machine vs OS Virtualization (containers)

Why Docker? Resource efficiency – shared OS Manage apps side-by-side in isolation – higher density Not a new concept (FreeBSD Jails, Solaris Zones,…) Easy container management Portable Ready-to-run containered applications Ability to create your own Integration into DevOps world Puppet, Chef, Vagrant, Ansible,… https://stackshare.io/posts/how-docker-fits-into-the-current-devops-landscape

Docker - Terminology Images – collection layered file system changes Does not have state and never changes Writeable container layer at the tope of a container Copy-on-write when a modification is made to an image layer, stores in writeable container layer https://developers.redhat.com/blog/2016/01/13/a-practical-introduction-to-docker-container-terminology/ https://docs.docker.com/glossary/ https://docs.docker.com/engine/userguide/networking/

Docker - Terminology Containers – runtime instance of an image Layered filesystem Data Volumes - persisted Network port bindings https://developers.redhat.com/blog/2016/01/13/a-practical-introduction-to-docker-container-terminology/ https://docs.docker.com/glossary/ https://docs.docker.com/engine/userguide/networking/

Docker - Terminology Registry – hosting service of images DockerHub – public registry 100,000+ apps on Docker Hub https://developers.redhat.com/blog/2016/01/13/a-practical-introduction-to-docker-container-terminology/ https://docs.docker.com/glossary/ https://docs.docker.com/engine/userguide/networking/

SQL Server Docker - Images Two different base-images: Linux and Windows Github - https://github.com/Microsoft/mssql-docker Requirements Docker Engine 1.8+ Min 4 GB disk Min 4 GB memory https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-docker

SQL Server Docker - Demo Docker commands Pull mssql-server-windows image Create a container, mapping port and data volume Restore a database Create more containers, using “attach_dbs” https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-docker https://www.sqlshack.com/running-sql-server-containers-windows-server-2016-core/

SQL Server Docker – Administration Management tooling options: Kitematic – Desktop UI interface to Hub Rancher - Orchestration Shipyard – Cluster management Deployment: Windows Server 2016 Azure – PaaS and IaaS options Resource allocation: CPU shares Specific cores Memory limit https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-docker https://www.sqlshack.com/running-sql-server-containers-windows-server-2016-core/

SQL Server Docker – Environment Variables https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-environment-variables

SQL Server 2017 - What else is new? Lots of Community driven enhancements! Smarter backup information – sys.dm_db_file_space_usage has modified_page counts SELECT INTO … ON FILEGROUP! – keep your staging separate Tempdb per-user database usage information VLF DMF - sys.dm_db_log_info() Sockets, cores per, numa_node_count - sys.dm_os_sys_info Improved small database backup performance https://getpocket.com/a/read/1716835032

What else is new? – Part 2 Resumeable online index rebuilds – pause, continue Graph Processing – represent complex relationships with NODES and EDGE tables, and the new MATCH operator LOB support in Clustered Columnstore indexes Adaptive JOINS – dynamic switching of joins during query execution Query store wait stats – query level wait information Automatic Tuning – query store analysis and automation to recommend and fix identified problems https://getpocket.com/a/read/1716835032

What else is new? – Part 3 Reporting Services Stand-alone install – no longer within the DB engine bits Power BI hosted reports Analysis Services Ragged Hierarchy members Detail row – custom row sets definable in the drill-down Tabular table-level security Machine Learning (formerly R Services) Python language support

What else is new? – Part 4 New Microsoft open source projects mssql-scripter – cross-platform SQL scripting tool ~= Import/Export wizards in SSMS DBFS - open-source FUSE mount of SQL DMV’s ??? – cross-platform management UI – coming soon sql-xplat-cli https://github.com/Microsoft/dbfs

What else is new? – Demo T-SQL Fun! TRIM() DBFS - open-source FUSE mount of SQL DMV’s ??? – cross-platform management UI – coming soon sql-xplat-cli https://github.com/Microsoft/dbfs

What else is new? - Demo T-SQL fun! BULK INSERT … FORMAT = ‘CSV’ TRIM(), TRANSLATE(), CONCAT_WS() http://files.grouplens.org/datasets/movielens/ml-latest-README.html MovieLens dataset - https://grouplens.org/datasets/movielens/ Full: 24,000,000 ratings and 670,000 tag applications applied to 40,000 movies by 260,000 users. Includes tag genome data with 12 million relevance scores across 1,100 tags. Last updated 10/2016.

Need Help? Check the Release Notes Post questions Stack Overflow (tag sql-server) MSDN Forums Search/File Bugs Microsoft Connect – report bugs/request features http://stackoverflow.com/questions/tagged/sql-server https://social.msdn.microsoft.com/Forums/en-US/home?category=sqlserver https://connect.microsoft.com/SQLServer/Feedback

Got a question? Andrew Loree andyloree.com andy@andyloree.com @LowOnDiskSpace