Using pgTap to unit test your functions and queries By Lloyd Albin pgTAP 1/3/2017

pgTAP pgTAP 1/3/2017

pgTAP: Unit Testing for PostgreSQL pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions. The TAP output is suitable for harvesting, analysis, and reporting by a TAP harness, such as those used in Perl applications. http://pgtap.org/ Good for: Application Development Schema Validation xUnit-Style Testing Module Development pgTAP 1/3/2017

pgTAP pgTAP 1/3/2017

Installing pg_tap Installation will require you to download the pgTAP source code and then compile and install pgTAP against your installed version of Postgres. It is also a good idea to install pg_prove. pgTAP 1/3/2017

Test Directory In our test directory we want to create some folders specific to the tests we are going to run. Create a common directory. Create a find_raster directory (Name of the function we are going to write.) mkdir common mkdir find_raster pgTAP 1/3/2017

Common Diagnostics pgTAP 1/3/2017

Common Diagnostics I like to create a common diagnostics that I run against all databases. common/diagnostic.sql -- Turn off echo and keep things quiet. \set ECHO \set QUIET 1 -- Format the output for nice TAP. \pset format unaligned \pset tuples_only true \pset pager -- Revert all changes on failure. \set ON_ERROR_ROLLBACK 1 \set ON_ERROR_STOP true \set QUIET 1 -- Begin the transaction. BEGIN; SET search_path TO public; pgTAP 1/3/2017

Set user to test as If you are not running as the owner of the database, switch to be the owner of the database. This is use full if you are testing as user postgres, otherwise this might be skipped. -- Inline function to set the role for extension installation DO $BODY$ DECLARE db_owner record; BEGIN     SELECT pg_user.usename INTO db_owner     FROM pg_database     LEFT JOIN pg_catalog.pg_user         ON pg_database.datdba = pg_user.usesysid     WHERE datname = current_database();     IF db_owner.usename <> current_user THEN         EXECUTE 'SET ROLE ' || db_owner.usename;         SET search_path TO public;     END IF; END $BODY$ LANGUAGE plpgsql; pgTAP 1/3/2017

Install pgtap extension We want to install the pgtap extension if it is not already installed. Because this is inside a transaction if the extension gets installed it will be automatically uninstalled at the end of the tests. Write certain psql variables to a temp table to test against. -- Install the TAP functions if it is not already installed. CREATE EXTENSION IF NOT EXISTS pgtap; -- Set the role to the user you wish to run the tests as. CREATE TEMP TABLE __pgtap_db_server__ (server text, username text, production_database text); INSERT INTO __pgtap_db_server__ (server, username, production_database) VALUES (:'HOST', :'test_user', :'test_production_database'); pgTAP 1/3/2017

pg_prove /usr/local/apps/perl/perl- current/bin/pg_prove -h sqltest-alt -d sandbox --recurse -v --ext sql find_raster DO $BODY$ DECLARE     server_name record; BEGIN     SELECT server, username, production_database INTO server_name FROM __pgtap_db_server__;     IF server_name.production_database = current_database() THEN         -- If production database, run as the owner of the database PERFORM 'SET ROLE ' || server_name.username;         SET search_path TO public;         --SELECT diag('Running on a production database'); ELSE         -- If not a production database, run as the executing user aka developer RESET ROLE;         SET search_path TO public;     END IF; END $BODY$ LANGUAGE plpgsql; pgTAP 1/3/2017

Validation Information We need to say how many tests we are going to perform. The diag command lets us generate output that is not a test so that we can show current variables, database, database version, timestamp, etc that is required for validation. -- Plan the tests. SELECT plan(:plan + 16); -- Configuration Data SELECT diag('Configuration'); SELECT diag('================================='); SELECT diag('Test Name: ' || :'test_name'); SELECT diag('Date: ' || current_timestamp); SELECT diag('Current Server: ' || :'HOST'); SELECT diag('Current Database: ' || current_database()); SELECT diag('Current Port: ' || :'PORT'); SELECT diag(''); SELECT diag('Current Session User: ' || session_user); SELECT diag('Current User: ' || current_user); SELECT diag('pgTAP Version: ' || pgtap_version()); SELECT diag('pgTAP Postgres Version: ' || pg_version()); SELECT diag('Postgres Version: ' || current_setting( 'server_version')); SELECT diag('OS: ' || os_name()); SELECT diag(''); SELECT diag('Common Tests'); SELECT diag('================================='); pgTAP 1/3/2017

Checking extensions Ok tests true/false Is test equals SELECT ok( :boolean, :description ); Is test equals SELECT is( :have, :want, :description ); SELECT ok((SELECT CASE WHEN current_setting( 'server_version_num') = pg_version_num()::text     THEN TRUE     ELSE FALSE     END), 'pgTAP is compiled against the correct Postgres Version'); SELECT is(         (SELECT extname FROM pg_catalog.pg_extension WHERE extname = 'plpgsql')     , 'plpgsql', 'Verifying extension plpgsql is installed'); SELECT is(         (SELECT extname FROM pg_catalog.pg_extension WHERE extname = 'plperl')     , 'plperl', 'Verifying extension plperl is installed'); SELECT is(         (SELECT extname FROM pg_catalog.pg_extension WHERE extname = 'pgtap')     , 'pgtap', 'Verifying extension pgtap is installed'); pgTAP 1/3/2017

Testing Optional Extensions SELECT CASE     WHEN current_database() = 'postgres'     THEN collect_tap(         is(                 (SELECT extname FROM pg_catalog.pg_extension WHERE extname = 'adminpack')             , 'adminpack', 'Verifying extension adminpack is installed'),         is(                 (SELECT extname FROM pg_catalog.pg_extension WHERE extname = 'pg_buffercache')             , 'pg_buffercache', 'Verifying extension pgbuffercache is installed'),         is(                 (SELECT count(*)::int FROM pg_catalog.pg_extension)             , 5, 'Verifying only 5 extensions are installed')     )     ELSE collect_tap(         skip('Skipping extenstion test for adminpack', 1),         skip('Skipping extenstion test for pg_buffercache', 1),         is(                 (SELECT count(*)::int FROM pg_catalog.pg_extension)             , 3, 'Verifying only 3 extensions are installed')     )     END; If we are in the postgres database we want to check for two more extensions otherwise we want to skip testing for those extensions. collect_tap performs multiple tests at once. SELECT collect_tap(:lines); Skip allows you to skip tests that you have allotted with plan(x). SELECT skip( :why, :how_many ); pgTAP 1/3/2017

Testing for Extra Extensions This will list all the extra extensions installed. SELECT diag('Extra Extension Installed: ' || extname) FROM pg_catalog.pg_extension     WHERE (         current_database() = 'postgres'         AND extname != 'plpgsql'         AND extname != 'plperl'         AND extname != 'pgtap'         AND extname != 'adminpack'         AND extname != 'pg_buffercache')         OR (         current_database() <> 'postgres'         AND extname != 'plpgsql'         AND extname != 'plperl'         AND extname != 'pgtap'); pgTAP 1/3/2017

Testing Languages installed We are going to test that we have the correct languages installed and make sure that we do not have the un- secure perl installed. has_language checks for the installed langauge. SELECT has_language( :language ); hasnt_language check to make sure it is not installed. SELECT hasnt_language( :language ); SELECT has_language( 'c' ); SELECT has_language( 'internal' ); SELECT has_language( 'sql' ); SELECT has_language( 'plpgsql' ); SELECT has_language( 'plperl' ); SELECT hasnt_language( 'plperlu' ); pgTAP 1/3/2017

Testing for Extra Languages This test will fail if we have extra languages installed and then list the extra languages. SELECT is(         (SELECT count(*)::int FROM pg_catalog.pg_language)     , 5, 'Verifying no extra languages are installed'); SELECT diag('Extra Languages Installed: ' || lanname) FROM pg_catalog.pg_language     WHERE lanname != 'c'         AND lanname != 'internal'         AND lanname != 'sql'         AND lanname != 'plpgsql'         AND lanname != 'plperl'; pgTAP 1/3/2017

Testing for perl bug. We once ran into a bug with perl when upgrading. This tests that multiplicity is defined within the perl otherwise perl will not work properly with Postgres. lives_ok tests a prepared statement or sql and makes sure that it does not error out. SELECT lives_ok( :sql, :description ); SET ROLE postgres; SET search_path TO public; CREATE FUNCTION public.perl_test () RETURNS integer AS $body$ return 1; $body$ LANGUAGE 'plperl' VOLATILE CALLED ON NULL INPUT SECURITY INVOKER; GRANT EXECUTE   ON FUNCTION public.perl_test() TO PUBLIC; PREPARE dba_perl_test AS SELECT * FROM public.perl_test(); SELECT lives_ok('dba_perl_test','Testing plperl has multiplicity defined - Test 1'); SET ROLE dba; SET search_path TO public; SELECT lives_ok('dba_perl_test','Testing plperl has multiplicity defined - Test 2'); RESET ROLE; pgTAP 1/3/2017

Change back to our test user and start the tests Here we set the role back to our testing user and get ready to start testing. DO $BODY$ DECLARE     server_name record; BEGIN     SELECT server, username, production_database INTO server_name FROM __pgtap_db_server__;     IF server_name.production_database = current_database() THEN         -- If production database, run as the owner of the database PERFORM 'SET ROLE ' || server_name.username;         --SELECT diag('Running on a production database'); ELSE         -- If not a production database, run as the executing user aka developer RESET ROLE;     END IF; END $BODY$ LANGUAGE plpgsql; SELECT diag('Tests'); SELECT diag('================================='); pgTAP 1/3/2017

Running the basic tests pgTAP 1/3/2017

pg_prove Inside our find_raster directory we want to create a 01_find_raster.sql file. This file right now is only going to test our common diagnostics file. -- Setup Test Variables \set test_name 'find_raster' \set test_user 'lalbin' \set test_production_database 'sandbox' \set plan 0 -- Install pgTAP, show diagnostics, and start common tests \ir ../common/diagnostic.sql pgTAP 1/3/2017

pg_prove pg_prove -h host -d database --recurse Follow the directories recursively -v Verbose mode, shows you each test --ext sql The extension of the files you will be using. find_raster The name of the directory to start with. /usr/local/apps/perl/perl-current/bin/pg_prove -h sqltest-alt -d sandbox --recurse -v --ext sql find_raster pgTAP 1/3/2017

Running our test Here is the general information for validation. 1..16 # Configuration # ================================= # Test Name: find_raster # Date: 2017-01-03 15:26:37.274103-08 # Current Server: sqltest-alt # Current Database: sandbox # Current Port: 5432 # # Current Session User: postgres # Current User: postgres # pgTAP Version: 0.97 # pgTAP Postgres Version: 9.4.9 # Postgres Version: 9.4.9 # OS: linux # pgTAP 1/3/2017

Running our test Here are the results of our general testing. # Common Tests # ================================= ok 1 - pgTAP is compiled against the correct Postgres Version ok 2 - Verifying extension plpgsql is installed ok 3 - Verifying extension plperl is installed ok 4 - Verifying extension pgtap is installed ok 5 # SKIP Skipping extenstion test for adminpack ok 6 # SKIP Skipping extenstion test for pg_buffercache ok 7 - Verifying only 3 extensions are installed ok 8 - Procedural language c should exist ok 9 - Procedural language internal should exist ok 10 - Procedural language sql should exist ok 11 - Procedural language plpgsql should exist ok 12 - Procedural language plperl should exist ok 13 - Procedural language plperlu should not exist ok 14 - Verifying no extra languages are installed ok 15 - Testing plperl has multiplicity defined - Test 1 ok 16 - Testing plperl has multiplicity defined - Test 2 pgTAP 1/3/2017

Running our test Now we started our find_raster testing, for which we have not yet written any tests. So everything passed ok so far. Now we are ready to starting writing out tests and function. # Tests # ================================= ok All tests successful. Files=1, Tests=16, 0 wallclock secs ( 0.02 usr 0.02 sys + 0.00 cusr 0.02 csys = 0.06 CPU) Result: PASS pgTAP 1/3/2017

find_raster pgTAP 1/3/2017

The Problem We receive faxes that are multi-page TIFF’s. The TIFF file name are called the raster id. We have data where we have the full path of the file name, the raster id and the raster id with page number. Examples: 0000/000000 0000/0000001111 /studydata/studyname/0000/000000 pgTAP 1/3/2017

Writing our tests We now change the plan to 3 tests. We write our basic three tests. Now we are ready to write our function. \set plan 3 -- Install pgTAP, show diagnostics, and start common tests \ir ../common/diagnostic.sql SELECT is((SELECT find_raster('1234/567890')), '1234/567890', 'Testing find_raster using Raster ID'); SELECT is((SELECT find_raster('1234/5678901234')), '1234/567890', 'Testing find_raster using Raster ID with Page Number'); SELECT is((SELECT find_raster('/study_data/study_name/1234/567890')), '1234/567890', 'Testing find_raster using Directory path with File Name'); pgTAP 1/3/2017

find_raster function The first thing to do, is to be able to find the Raster ID, no matter which format is supplied. IMMUTABLE is required to be able to use this function as part of an index. This function was written to fit on this page. Some of the other items to put inside the real function would be: We should check the return to make sure a ‘/’ is in the correct spot and through an error if it is not. “R” is also valid. If the raster is less then 23 characters to return an error, except for when it is 11 or 15 in length. CREATE FUNCTION find_raster (raster varchar) RETURNS VARCHAR(11) AS $$ BEGIN     CASE length(raster)         WHEN 11 THEN             -- Format: 1234/567890             -- Returns: 1234/567890 RETURN raster;         WHEN 15 THEN             -- Format: 1234/5678901234             -- Returns: 1234/567890 RETURN substr(raster, 1, 11);         ELSE             -- Format: /study_data/study_name/1234/567890             -- Returns: 1234/567890 RETURN substr(raster, length(raster)-10, 11);     END CASE; END; $$ LANGUAGE plpgsql IMMUTABLE RETURNS NULL ON NULL INPUT; pgTAP 1/3/2017

Testing our new function The first thing to do, is to be able to find the Raster ID, no matter which format is supplied. IMMUTABLE is required to be able to use this function as part of an index. This function was written to fit on this page. Some of the other items to put inside the real function would be: We should check the return to make sure a ‘/’ is in the correct spot and through an error if it is not. “R” is also valid. If the raster is less then 23 characters to return an error, except for when it is 11 or 15 in length. # Tests # ================================= ok 17 - Testing find_raster using Raster ID ok 18 - Testing find_raster using Raster ID with Page Number ok 19 - Testing find_raster using Directory path with File Name ok All tests successful. Files=1, Tests=19, 0 wallclock secs ( 0.05 usr 0.01 sys + 0.00 cusr 0.02 csys = 0.08 CPU) Result: PASS pgTAP 1/3/2017

Now we run across bad data Lets say the ‘/’ ended up being an ‘X’ so we need to make the function fail and test for that failure. But ‘R’ is also a valid character to we need to make sure the function does not break with an ‘R’. So we now need to update our tests and code. Update existing test comments to say ‘/’ and add test for ‘R’ and ‘X’. throws_ok allows us to test for a specific error message. SELECT throws_ok( :sql, :errmsg, :description ); \set plan 9 … SELECT is((SELECT find_raster('/study_data/study_name/1234R567890')), '1234R567890', 'Testing R find_raster using Directory path with File Name'); SELECT throws_ok(E'SELECT find_raster(''1234X567890'')', 'Invalid Character "X"', 'Testing X find_raster using Raster ID'); SELECT throws_ok(E'SELECT find_raster(''1234X5678901234'')', 'Invalid Character "X"', 'Testing X find_raster using Raster ID with Page Number'); SELECT throws_ok(E'SELECT find_raster(''/study_data/study_name/1234X567890'')', 'Invalid Character "X"', 'Testing X find_raster using Directory path with File Name'); pgTAP 1/3/2017

Now if we test before updating the code ok 17 - Testing / find_raster using Raster ID ok 18 - Testing / find_raster using Raster ID with Page Number ok 19 - Testing / find_raster using Directory path with File Name ok 20 - Testing R find_raster using Raster ID ok 21 - Testing R find_raster using Raster ID with Page Number ok 22 - Testing R find_raster using Directory path with File Name not ok 23 - Testing X find_raster using Raster ID # Failed test 23: "Testing X find_raster using Raster ID" # caught: no exception # wanted: an exception not ok 24 - Testing X find_raster using Raster ID with Page Number # Failed test 24: "Testing X find_raster using Raster ID with Page Number" # caught: no exception # wanted: an exception not ok 25 - Testing X find_raster using Directory path with File Name # Failed test 25: "Testing X find_raster using Directory path with File Name" # caught: no exception # wanted: an exception Failed 3/25 subtests         (less 2 skipped subtests: 20 okay) Test Summary Report ------------------- find_raster/01_find_raster.sql (Wstat: 0 Tests: 25 Failed: 3)   Failed tests: 23-25 Files=1, Tests=25, 1 wallclock secs ( 0.06 usr 0.01 sys + 0.00 cusr 0.02 csys = 0.09 CPU) Result: FAIL If we run the tests before we update the code, we will see how the find_raster function does not fail for the X condition. So now we need to fix the code and test again. pgTAP 1/3/2017

Update the function We now test for ‘/’ and ‘R’ and for any other character we raise an exception. Ready to test again. DECLARE   raster_id VARCHAR; BEGIN   CASE length(raster)     WHEN 11 THEN raster_id := raster;     WHEN 15 THEN raster_id := substr(raster, 1, 11);     ELSE raster_id := substr(raster, length(raster)-10, 11);   END CASE;   CASE substr(raster_id, 5, 1)     WHEN '/' THEN     WHEN 'R' THEN     ELSE       RAISE EXCEPTION 'Invalid Character "%"', substr(raster_id, 5, 1);   END CASE;   RETURN raster_id; END; pgTAP 1/3/2017

Testing with the fixed function. Now it passes all the test. Finding the good raster id’s and failing for the bad raster id’s. ok 17 - Testing / find_raster using Raster ID ok 18 - Testing / find_raster using Raster ID with Page Number ok 19 - Testing / find_raster using Directory path with File Name ok 20 - Testing R find_raster using Raster ID ok 21 - Testing R find_raster using Raster ID with Page Number ok 22 - Testing R find_raster using Directory path with File Name ok 23 - Testing X find_raster using Raster ID ok 24 - Testing X find_raster using Raster ID with Page Number ok 25 - Testing X find_raster using Directory path with File Name ok All tests successful. Files=1, Tests=25, 0 wallclock secs ( 0.03 usr 0.01 sys + 0.00 cusr 0.01 csys = 0.05 CPU) Result: PASS pgTAP 1/3/2017

Now lets test for bad lengths If the raster is less then 23 characters to return an error, except for when it is 11 or 15 in length. SELECT throws_ok(E'SELECT find_raster(''1234X5678'')', 'Invalid Raster Length: 9', 'Testing find_raster using bad Raster ID'); SELECT throws_ok(E'SELECT find_raster(''1234X56789012'')', 'Invalid Raster Length: 13', 'Testing find_raster using bad Raster ID with Page Number'); SELECT throws_ok(E'SELECT find_raster(''/a/z/1234X567890'')', 'Invalid Raster Length: 16', 'Testing find_raster using bad Directory path with File Name'); pgTAP 1/3/2017

Update the function, again If the raster is less then 23 characters to return an error, except for when it is 11 or 15 in length. BEGIN   CASE     WHEN length(raster) = 11 THEN     WHEN length(raster) = 15 THEN     WHEN length(raster) < 23 THEN       RAISE EXCEPTION 'Invalid Raster Length: %', length(raster);     ELSE   END CASE; pgTAP 1/3/2017

Testing with the fixed function. Now it passes all the test. Finding the good raster id’s and failing for the bad raster id’s. ok 17 - Testing / find_raster using Raster ID ok 18 - Testing / find_raster using Raster ID with Page Number ok 19 - Testing / find_raster using Directory path with File Name ok 20 - Testing R find_raster using Raster ID ok 21 - Testing R find_raster using Raster ID with Page Number ok 22 - Testing R find_raster using Directory path with File Name ok 23 - Testing X find_raster using Raster ID ok 24 - Testing X find_raster using Raster ID with Page Number ok 25 - Testing X find_raster using Directory path with File Name ok 26 - Testing find_raster using bad Raster ID ok 27 - Testing find_raster using bad Raster ID with Page Number ok 28 - Testing find_raster using bad Directory path with File Name ok All tests successful. Files=1, Tests=28, 0 wallclock secs ( 0.05 usr 0.03 sys + 0.01 cusr 0.02 csys = 0.11 CPU) Result: PASS pgTAP 1/3/2017

pgTAP and testing your queries 2/7/2017

Lets setup some tables Here we are setting up some tables to test with. CREATE TABLE public.table_a ( id SERIAL, name VARCHAR, PRIMARY KEY(id) ) WITH (oids = false); CREATE TABLE public.table_b ( id SERIAL, a_id INTEGER, status VARCHAR, action_dt TIMESTAMP WITHOUT TIME ZONE, PRIMARY KEY(id) ) WITH (oids = false); pgTAP 2/7/2017

Lets setup a poor view We are wanting the employee’s current status for all currently employed employees. CREATE OR REPLACE VIEW public.view_a AS  SELECT a.id,     a.name,     b.status,     b.action_dt    FROM table_a a      LEFT JOIN ( SELECT table_b.id,         table_b.a_id,         table_b.status,         table_b.action_dt        FROM table_b          LEFT JOIN ( SELECT max(table_b_1.id) AS id,             table_b_1.a_id            FROM table_b table_b_1           GROUP BY table_b_1.a_id) c USING (id, a_id)         WHERE c.id IS NOT NULL) b ON a.id = b.a_id   WHERE b.status != 'Left' ORDER BY a.id; pgTAP 2/7/2017

Creating the initial tests. pgTAP 2/7/2017

Truncate the tables (17) Make a directory called employee_test and create a file called 01_employee_test.sql First we want to truncate all our tables. While this could have been just a normal SQL command, we can wrap it inside a lives_ok to verify that it worked without any errors. -- Setup Test Variables \set test_name 'employee_status' \set test_user 'lalbin' \set test_production_database 'sandbox' \set plan 4 -- Install pgTAP, show diagnostics, and start common tests \ir ../common/diagnostic.sql --\i common/diagnostic.sql SELECT lives_ok('TRUNCATE table_a, table_b', 'Truncating tables in preperation for testing'); pgTAP 2/7/2017

Insert the test data (18 & 19) Next we need to insert our test records. SELECT lives_ok(E'INSERT INTO table_a VALUES (1, ''Lloyd''), (2, ''Judy''), (3, ''Gerald'')', 'Inserting Test Data into table_a'); SELECT lives_ok(E'INSERT INTO table_b VALUES (1, 1, ''Hired'', ''1/1/14''), (2, 2, ''Hired'', ''1/1/15''), (3, 3, ''Hired'', ''1/1/16''), (4, 2, ''Maternity'', ''4/5/16''), (5, 2, ''Normal'', ''5/5/16''), (6, 3, ''Left'', ''10/15/16'')', 'Inserting Test Data into table_b'); pgTAP 2/7/2017

Testing the data (20) Use results_eq to test the output of the view. By default the values will have a field type of “unknown” and that is why we need to cast the variables in the first row to match the query output. SELECT results_eq( :sql, :sql, :description ); SELECT results_eq(     'SELECT id, name, status, action_dt FROM view_a;',     $$VALUES         (1,'Lloyd'::varchar,'Hired'::varchar,'2014- 01-01'::timestamp),         (2,'Judy','Normal','2016-05-05')$$,     'Verify Data in view_a'); id name (Employee) 1 Lloyd 2 Judy 3 Gerald id a_id status action_dt 1 Hired 1/1/2014 2 1/1/2015 3 1/1/2016 4 Maternity 4/5/2016 5 Normal 5/5/2016 6 Left 10/15/2016 id name status action_dt 1 Lloyd Hired 1/1/2014 2 Judy Normal 5/5/2016 pgTAP 2/7/2017

Run the Initial Test We get the results we expect. # Tests # ================================= ok 17 - Truncating tables in preperation for testing ok 18 - Inserting Test Data into table_a ok 19 - Inserting Test Data into table_b ok 20 - Verify Data in view_a ok All tests successful. Files=1, Tests=20, 0 wallclock secs ( 0.06 usr 0.01 sys + 0.00 cusr 0.04 csys = 0.11 CPU) Result: PASS pgTAP 2/7/2017

Creating a failure test (21 & 22) Now we need to write a new test to cause the bug that we wish to test. This is caused by entering the data out of order. SELECT lives_ok(E'INSERT INTO table_b VALUES (7, 2, ''Normal'', ''10/1/16''), (8, 2, ''Family Leave'', ''9/1/16'')', 'Inserting 2nd Test Data into table_b'); SELECT results_eq(     'SELECT id, name, status, action_dt FROM view_a;',     $$VALUES         (1,'Lloyd'::varchar,'Hired'::varchar,'2014- 01-01'::timestamp),         (2,'Judy','Normal','2016-10-01')$$,     'Verify Data in view_a'); id a_id status action_dt 1 Hired 1/1/2014 2 1/1/2015 3 1/1/2016 4 Maternity 4/5/2016 5 Normal 5/5/2016 6 Left 10/15/2016 7 10/1/2016 8 Family Leave 9/1/2016 id name (Employee) 1 Lloyd 2 Judy 3 Gerald id name status action_dt 1 Lloyd Hired 1/1/2014 2 Judy Family Leave 9/1/2016 pgTAP 2/7/2017

Run the Failure Test # Tests # ================================= ok 17 - Truncating tables in preperation for testing ok 18 - Inserting Test Data into table_a ok 19 - Inserting Test Data into table_b ok 20 - Verify Data in view_a ok 21 - Inserting 2nd Test Data into table_b not ok 22 - Verify Data in view_a # Failed test 22: "Verify Data in view_a" # Results differ beginning at row 2: # have: (2,Judy,"Family Leave","2016-09-01 00:00:00") # want: (2,Judy,Normal,"2016-10-01 00:00:00") Failed 1/22 subtests         (less 2 skipped subtests: 19 okay) Test Summary Report ------------------- employee_test/01_employee_test.sql (Wstat: 0 Tests: 22 Failed: 1)   Failed test: 22 Files=1, Tests=22, 1 wallclock secs ( 0.06 usr 0.01 sys + 0.01 cusr 0.02 csys = 0.10 CPU) Result: FAIL Here we see where the view failed and the bad line vers the good line. pgTAP 2/7/2017

Refactoring the Query CREATE VIEW public.view_b AS SELECT a.id,     a.name,     b.status,     b.action_dt FROM table_a a      LEFT JOIN (     SELECT DISTINCT ON (table_b.a_id) table_b.id,             table_b.a_id,             table_b.status,             table_b.action_dt     FROM table_b     ORDER BY table_b.a_id, table_b.action_dt DESC     ) b ON a.id = b.a_id WHERE b.status != 'Left' ORDER BY a.id; Re-factoring the query to have the correct status and at the same time switching to a higher performing query, as long as you create the index needed by the ORDER BY statement. After some use, you notice that this query starts slowing down and that sometimes when records are entered out of order that the status is wrong. So now we need to re-factor this query, but we also need to validate the output of the query to make sure we do not write a new bad query. id name status action_dt 1 Lloyd Hired 1/1/2014 2 Judy Family Leave 9/1/2016 pgTAP 2/7/2017

Replace test (22) SELECT results_eq(     'SELECT id, name, status, action_dt FROM view_b;',     $$VALUES         (1,'Lloyd'::varchar,'Hired'::varchar,'2014-01- 01'::timestamp),         (2,'Judy','Normal','2016-05-05')$$,     'Verify Data in view_b'); Copy test 20 and update sql from using view_a to using view_b. This will make sure with the basic data that we are still getting the same result with our re-factored view. pgTAP 2/7/2017

Adding new test (23 & 24) SELECT results_ne(     'SELECT id, name, status, action_dt FROM view_a;',     $$VALUES         (1,'Lloyd'::varchar,'Hired'::varchar,'2014-01- 01'::timestamp),         (2,'Judy','Normal','2016-10-01')$$,     'Verify Bad Data in view_a (2nd run)'); SELECT results_eq(     'SELECT id, name, status, action_dt FROM view_b;',     $$VALUES         (1,'Lloyd'::varchar,'Hired'::varchar,'2014-01- 01'::timestamp),         (2,'Judy','Normal','2016-10-01')$$,     'Verify Good Data in view_b (2nd run)'); First we need to copy the second test for view_a and make it use view_b. The we need to alter the second test for view_a to be a not equal test by using the results_ne command. SELECT results_ne( :sql, :sql, :description ); This will make sure with the more advanced test failed properly with view_a and works properly with view_b. pgTAP 2/7/2017

Running our new tests. # Tests # ================================= ok 17 - Truncating tables in preperation for testing ok 18 - Inserting Test Data into table_a ok 19 - Inserting Test Data into table_b ok 20 - Verify Data in view_a ok 21 - Verify Data in view_b ok 22 - Inserting 2nd Test Data into table_b ok 23 - Verify Bad Data in view_a (2nd run) ok 24 - Verify Good Data in view_b (2nd run) ok All tests successful. Files=1, Tests=24, 0 wallclock secs ( 0.04 usr 0.03 sys + 0.01 cusr 0.03 csys = 0.11 CPU) Result: PASS Now we see that the old view fails like we expected and the new view works like we also expected. pgTAP 2/7/2017

Upcomming Conferences pgTAP 2/7/2017

PostgreSQL@SAaLE15x – March 2017 https://www.socallinuxexpo.org/scale/15x/postgresqlscale March 2nd & 3rd (Thursday & Friday) PostgreSQL Track March 2nd – 5th (Thursday – Sunday) Southern California Linux Expo Pasadena Convention Center, Pasadena, California $85 SCALE Pass (Provides access to all SCALE sessions, the exhibit hall, and free events.) Currently: Registration Open pgTAP 2/7/2017

PGConf US – March 2017 http://www.pgconf.us/conferences/2017 March 28th (Tuesday) Training / Enterprise March 29th – 31st (Wednesday – Friday) PostgreSQL Track Westin Jersey City, Jersey City, New Jersey $573.42 Regular Admission $209.94 Training Classes (Morning or Afternoon) $104.94 Enterprise Classes Currently: Registration Open pgTAP 2/7/2017

LinuxFest Northwest – May 2017 https://www.linuxfestnorthwest.org/ May 6th & 7th (Saturday & Sunday) Bellingham Technical College, Bellingham, Washington Free Currently: Call for Papers Postgres on Saturday Only? PNW PgDay instead of Postgres Track? https://www.meetup.com/Whatcom-PostgreSQL pgTAP 2/7/2017

PGCon – May 2017 https://www.pgcon.org/ Tutorials May 23rd & 24th (Tuesday & Wednesday) Developer Unconference May 24th (Wednesday) Talks May 25th & 26th (Thursday & Friday) User Unconference May 27th (Saturday) University of Ottawa, Ottawa, Ontario, Canada $195 Individual / $350 Corporate / $60 Student (Unconference & Talks) $75 per Tutorial (1/2 day) (Tutorials) Currently: Selecting Presentations / Registration Opens 3/29 ish. pgTAP 2/7/2017

Postgres Open– September 2017 https://2017.postgresopen.org/ Tutorials September 6th (Wednesday) Talks September 7th & 8th (Thursday & Friday) PostgresOpen Golf Charity Event (Friday) - https://2016.postgresopen.org/golf/ PARC 55 Hilton Hotel, San Francisco, California $350 Early Bird / $550 Regular (Talks) (2016 Prices) $200 per Tutorial (Tutorials) (2016 Prices) $100 Golf Charity Event / $50 Club Rental (Optional) (2016 Prices) Currently: Call for Papers starts in April pgTAP 2/7/2017