How to corrupt your data by accident BY: LLOYD ALBIN 9/3/2013

Create test databases The first thing we need to do is to create two test databases for our example. createdb -h sqltest lloyd_test_1 createdb -h sqltest lloyd_test_2

Creating our test tables Here we are creating our test tables. We can see that there is a possible foreign key relationship between the tables using the network column. CREATE TABLE public.table1 ( network VARCHAR, PRIMARY KEY(network) ) ; CREATE TABLE public.table2 ( pk SERIAL, network VARCHAR, PRIMARY KEY(pk) ) ;

Creating the foreign key relationship Here is our foreign key relationship. ALTER TABLE public.table2 ADD CONSTRAINT table2_fk FOREIGN KEY (network) REFERENCES public.table1(network) ON DELETE NO ACTION ON UPDATE CASCADE NOT DEFERRABLE;

Inserting some test data Here is the test data. As you can see we have created two records in each table. We are using the foreign key relationship twice, once for ‘TEST’ and once for ‘CROSS’. INSERT INTO public.table1 VALUES ('CROSS'); INSERT INTO public.table1 VALUES ('TEST'); INSERT INTO public.table2 (network) VALUES ('CROSS'); INSERT INTO public.table2 (network) VALUES ('TEST');

Testing Deletion The deletion fails just as expected, due to the foreign key relationship. DELETE FROM public.table1 WHERE network = 'CROSS'; ERROR: update or delete on table "table1" violates foreign key constraint "table2_fk" on table "table2" DETAIL: Key (network)=(CROSS) is still referenced from table "table2".

Creating the problem We have a sync process that can delete data even though there is a foreign key constraint. psql –h sqltest –d lloyd_test_1 –f delete.sql BEGIN … DELETE 1 … COMMIT

The Data (Table 2) We can see that we have two records and these records have foreign key relationships to table1, so there MUST be matching data in table1. SELECT * FROM table2; pk | network | CROSS 2 | TEST (2 rows)

The Data (Table 1) What ???? Where is our CROSS network? We have deleted it violating the foreign key relationship. SELECT * FROM table1; network TEST (1 row)

Dependencies As you can see all the dependencies are still in place.

Backup The backup’s won’t complain at all, so you would not even know that there is anything wrong. pg_dump -h sqltest -Fp lloyd_test_1 > lloyd_test_1.sql pg_dump -h sqltest -Fc lloyd_test_1 > lloyd_test_1.dump

Restore via psql When you try and restore the database, this is when you will find out there is a problem. It can’t implement the foreign key relationship because of the row that we deleted. From this, you might think that it was inserting data, but it was really applying the foreign key relationship. psql -h sqltest -d lloyd_test_2 -f lloyd_test_1.sql … psql:lloyd_test_1.sql:129: ERROR: insert or update on table "table2" violates foreign key constraint "table2_fk" DETAIL: Key (network)=(CROSS) is not present in table "table1". …

Restore via pg_restore When you try and restore the database, this is when you will find out there is a problem. It can’t implement the foreign key relationship because of the row that we deleted. With pg_restore, you can tell for sure that it failed during the creation of the foreign key relationship. pg_restore -h sqltest -d lloyd_test_2 lloyd_test_1.dump pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 2526; FK CONSTRAINT table2_fk postgres pg_restore: [archiver (db)] could not execute query: ERROR: insert or update on table "table2" violates foreign key constraint "table2_fk" DETAIL: Key (network)=(CROSS) is not present in table "table1". Command was: ALTER TABLE ONLY table2 ADD CONSTRAINT table2_fk FOREIGN KEY (network) REFERENCES table1(network) ON UPDATE CASCADE; WARNING: errors ignored on restore: 1

Hands on time Feel free to look at this database personally at this point so that you can personally see that the data is violating the foreign key constraint.

delete.sql Disables the internally created constraint triggers used by the foreign key relationship and then deletes the row of data. The COMMIT fails to check the constraints. BEGIN; SET session_replication_role = 'replica'; DELETE FROM public.table1 WHERE "network" = E'CROSS'; COMMIT;

session_replication_role Here is the documentation on this command. Please note that there is nothing said about foreign key’s. org/docs/9.2/static/run time-config- client.html#GUC- SESSION- REPLICATION-ROLE session_replication_role (enum) Controls firing of replication-related triggers and rules for the current session. Setting this variable requires superuser privilege and results in discarding any previously cached query plans. Possible values are origin (the default), replica and local. See ALTER TABLE for more information.

ALTER TABLE - TRIGGER DISABLE/ENABLE [ REPLICA | ALWAYS ] TRIGGER These forms configure the firing of trigger(s) belonging to the table. A disabled trigger is still known to the system, but is not executed when its triggering event occurs. For a deferred trigger, the enable status is checked when the event occurs, not when the trigger function is actually executed. One can disable or enable a single trigger specified by name, or all triggers on the table, or only user triggers (this option excludes internally generated constraint triggers such as those that are used to implement foreign key constraints or deferrable uniqueness and exclusion constraints). Disabling or enabling internally generated constraint triggers requires superuser privileges; it should be done with caution since of course the integrity of the constraint cannot be guaranteed if the triggers are not executed. The trigger firing mechanism is also affected by the configuration variable session_replication_role. Simply enabled triggers will fire when the replication role is "origin" (the default) or "local". Triggers configured as ENABLE REPLICA will only fire if the session is in "replica" mode, and triggers configured as ENABLE ALWAYS will fire regardless of the current replication mode.

ALTER TABLE - RULE Rule based items. org/docs/9.2/static/sql- altertable.html DISABLE/ENABLE [ REPLICA | ALWAYS ] RULE These forms configure the firing of rewrite rules belonging to the table. A disabled rule is still known to the system, but is not applied during query rewriting. The semantics are as for disabled/enabled triggers. This configuration is ignored for ON SELECT rules, which are always applied in order to keep views working even if the current session is in a non- default replication role.

How I read the docs To me, none of the documentation talked about being able to bypass the foreign key relationships. Especially when you read the ALTER TABLE - TRIGGER information that says: “this option excludes internally generated constraint triggers such as those that are used to implement foreign key constraints or deferrable uniqueness and exclusion constraints”

ENABLE/DISABLE TRIGGER ALL Here is another way to cause the same problem. It would be nice if they re-checked the foreign key’s if they were disabled and re- enabled during a transaction. BEGIN; ALTER TABLE public.table1 DISABLE TRIGGER ALL; DELETE FROM public.table1 WHERE network = E'CROSS'; ALTER TABLE public.table1 ENABLE TRIGGER ALL; COMMIT;

Drop/Create Foreign Key While your application could drop and re-create the foreign key. The drop of the foreign key will be stalled until everyone has stopped using any tables related to the foreign key.