Practical Session 13 Persistence Layer

Persistence Layer Design pattern for managing storing and accessing of permanent data Manages communication between the application and its database Effectively creates a separation between the application logic and the database access Better code stability Better usability SQL code reuse - queries/commands are written only once

Persistence Layer Components Data Transfer Object – DTO An object that represents a record from a single table. Its variables represent the columns of the table. Data Access Object – DAO Contain methods for retrieving and storing DTOs. Each DAO is responsible for a single DTO. Repository Contains commands/queries that span over multiple tables. Effectively manages multiple related DTOs

Data Transfer Object DTOs are passed to and from the persistence layer. If transferred to application logic: They contain the data retrieved from the database. Contains query result. If transferred to persistence layer They contain the data that to be written to the database. Data will added using commands. Naming Convention: DTO name will be the singular of a plural table name. Example: Table named “grades”, DTO will be named “grade” Names of DTO constructor parameters == DTO fields names == table represented by the DTO column names Each DTO class represents a single table.

Implementation Example Students Contains student information Fields: name Questions Contains the questions information Fields: question, answer Grades: Contains grade for each student and his questions Fields: student, question, grade

DTO Implementation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 class Student(object): def __init__(self, id, name): self.id = id self.name = name class Question(object): def __init__(self, num, answer): self.num = num self.answer = answer class Grade(object): def __init__(self, student_id, question_num, grade): self.student_id = student_id self.question_num = question_num self.grade = grade

Student - DAO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 class _Students(object): #Underscore means singleton def __init__(self, conn): self._conn = conn def insert(self, student): self._conn.execute(""" INSERT INTO students (id, name) VALUES (?, ?) """, [student.id, student.name]) def find(self, student_id): c = self._conn.cursor() c.execute(""" SELECT id, name FROM students WHERE id = ? """, [student_id]) return Student(*c.fetchone())

Questions - DAO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 class _Questions(object): def __init__(self, conn): self._conn = conn def insert(self, question): self._conn.execute(""" INSERT INTO questions (num, answer) VALUES (?, ?) """, [question.num, question.answer]) def find(self, num): c = self._conn.cursor() c.execute(""" SELECT num,answer FROM questions WHERE num = ? """, [num]) return Question(*c.fetchone())

Grades - DAO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 class _Grades(object): def __init__(self, conn): self._conn = conn def insert(self, grade): self._conn.execute(""" INSERT INTO grades (student_id, question_num, grade) VALUES (?, ?, ?) """, [grade.student_id, grade.question_num, grade.grade]) def find_all(self): c = self._conn.cursor() all = c.execute(""" SELECT student_id, question_num, grade FROM grades """).fetchall() return [Grade(*row) for row in all]

Repository 1 2 3 4 5 6 7 8 9 class _Repository(object): def __init__(self): self._conn = sqlite3.connect('grades.db') self.students = _Students(self._conn) self.questions = _Questions(self._conn) self.grades = _Grades(self._conn) def _close(self): self._conn.commit() self._conn.close()

Repository - Continued 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 def create_tables(self): _conn.executescript(""“ CREATE TABLE students ( id INT PRIMARY KEY, name TEXT NOT NULL ); CREATE TABLE questions ( num INT PRIMARY KEY, answer TEXT NOT NULL ); CREATE TABLE grades ( student_id INT NOT NULL, question_num INT NOT NULL, grade INT NOT NULL, FOREIGN KEY(student_id) REFERENCES students(id), FOREIGN KEY(question_num) REFERENCES questions(num), PRIMARY KEY (student_id, question_num) ); ""“)

Application Logic- Adding Grades 1 2 3 4 5 6 7 8 9 10 11 12 13 def grade(questions_dir, question_num): answer = repo. questions.find(question_num).answer for question in os.listdir(questions_dir): (student_id, ext) = os.path.splitext(question) code = imp.load_source('test', questions_dir + '/' + question) student_grade = Grade(student_id, question_num, 0) if code.run_question() == answer: student_grade.grade = 100 repo.grades.insert(student_grade)

Application Logic – Printing Grades 1 2 3 4 5 6 7 def print_grades(): print 'grades:' for grade in repo.grades.find_all(): student = repo.students.find(grade.student_id) print 'grade of student {} on assignment {} is {}'\ .format(student.name, grade.assignment_num, grade.grade) This method goes over all the students to find the name of the student of a specific grades This is not an efficient solution. A better solution would use JOIN. Next, we add JOIN support to our later.

Adding JOIN to the Repository 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 class StudentGradeWithName(object): def __init__(self, name, question_num, grade): self.name = name self. question_num = question_num self.grade = grade #this function is added to the Repository def get_grades_with_names(self): c = self._conn.cursor() all = c.execute(""" SELECT students.name, grades.assignment_num, grades.grade FROM grades JOIN students ON grades.student_id = students.student_id """).fetchall() return [StudentGradeWithName(*row) for row in all]

Updating the Application Logic 1 2 3 4 567 def print_grades(): print 'grades:' for studentGradeWithName in repo.get_grades_with_names(): print 'grade of student {} on assignment {} is {}'\ .format(studentGradeWithName.name, studentGradeWithName.assignment_num, studentGradeWithName.grade)

ORM - Object Relational Mapping What if we wish to update a grade? What if we wish to update a student name? Should we make an update function for each case? Solution: Generic functions. ORM method allows mapping between a DTO and its table. Using ORM we can implement a generic DAO containing: Generic delete Generic update

ORM Implementation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 import inspect #allows reaching constructor arguments def orm(cursor, dto_type): #the following line retrieve the argument names of the constructor args = inspect.getargspec(dto_type.__init__).args #__init__ == constructor args = args[1:] #args[0] == ‘self’, so we ignore it #gets the names of the columns returned in the cursor, after SELECT was executed col_names = [column[0] for column in cursor.description] #map them into the position of the corresponding constructor argument col_mapping = [col_names.index(arg) for arg in args] return [row_map(row, col_mapping, dto_type) for row in cursor.fetchall()] def row_map(row, col_mapping, dto_type): ctor_args = [row[idx] for idx in col_mapping] return dto_type(*ctor_args)

Generic DOA implementation Code @ website

Generic Delete - DAO 1 2 3 4 5 6 7 8 9 def delete(self, **keyvals): column_names = keyvals.keys() params = keyvals.values() stmt = 'DELETE FROM {} WHERE {}' \ .format(self._table_name, ' AND '.join([col + '=?' for col in column_names])) c = self._conn.cursor() c.execute(stmt, params)

Generic Update - DAO 1 2 3 4 5 6 7 8 9 10 11 12 13 14 def update(self, set_values, cond): set_column_names = ','.join(set_values.keys()) set_params = set_values.values() cond_column_names = ','.join(cond.keys()) cond_params = cond.values() params = set_params + cond_params stmt = 'UPDATE {} SET ({}) WHERE ({})'\ .format(self._table_name, ' AND '.join([set + '=?' for set in set_column_names]), ' AND '.join([cond + '=?' for cond in cond_column_names])) self._conn.execute(stmt, params).