1. EXTENDING PYTHON SCRIPTING I/III References: https://docs.python.org/3.4/c-api/ https://docs.python.org/3.4/extending/index.html#extending-index

2. GOALS Part I: Create a module (a C dll), callable from python [ extension ] Functions Class (with methods) Illustrate type conversions Introduce reference counting Part II: Create a python interpreter embedded in a C program. Coordinate python scrips and C functionality Part III: [?] Embed into our larger project. Make a (partial) mirror of our C GameObject in Python.

3. PYTHON C API OVERVIEW A massive collection of C (not C++) functions Naming conventions: Py_XXX: where XXX is a top-level function PyYYY_ZZZ: where YYY is a python type and ZZZ is the method. E.g. PyTuple_GetSize PyObject: nearly everything uses (and is one of) these primitives: int, floats complex primitives: strings, lists, tuples functions classes (and instances) modules exceptions You have to check the type (PyYYY_Check(PyObject*)). A lot of our work involves converting C Python data

4. FIRST STEPS DLL release 1 project (I'm calling mine testMod ) Include directories = "%PythonHome\include" Library directories = "%PythonHome\libs" Input is "Python3.lib" (more limited, but usable on python 3.x) …or "Python33.lib" (has any 3.3-specific functionality) Change name of extension to pyd Put any test python code next to the.pyd file. In this section, we'll run python code with the normal interpreter (PyCharm, Idle, cmd-line all OK) C code Python code 1. Unless you build python from source, you're using a release build (so our dll has to be release as well)

5. PART I: MODULE CREATIONS + FUNCTION Warning : I know you could just zip through this lecture and copy- paste. I suggest, however, that you do it with us – I'll try to add running commentary as we go that will (hopefully) help you better understand it. Ask questions as we go through this!!

6. CREATING THE MODULE #include static struct PyModuleDef testMod_definition = { PyModuleDef_HEAD_INIT, "testMod", "My super-awesome first python extension", -1, NULL }; PyMODINIT_FUNC PyInit_testMod(void) { PyObject * mod = PyModule_Create(&testMod_definition); return mod; }

7. TESTING THE MODULE import testMod print("testMod.__name__ = " + testMod.__name__) print("testMod.__doc__ = " + testMod.__doc__) print(dir(testMod))

8. TANGENT: REFERENCE-COUNTING AND GARBAGE COLLECTION INTRO L = [["a", 4.7], ["b", 1.1]]# RC on all are 1 L[0] = ["c", 9.2] # RC on "a" dropped to # 0 – GC G = L# RC on main list is 2 H = L[1] # RC on "b" is 2 L = "hi!" # RC on main list is 1 G = "bye!" # RC on main list is 0 – GC # "b" has RC of 1 # so no GC H = "hasta la vista"# Now "b" has RC of 0 - GC

9. ADDING CONSTANTS (For example math.pi in the built-in math module) PyObject * mod = PyModule_Create(&testMod_definition); if (mod == NULL) return NULL; // Add a (totally pointless) integer constant to the module // You'll need PyModule_??? [Finish me] import testMod print("testMod.mysteryValue = " + str(testMod.mysteryValue))

10. ADDING A FUNCTION (GOAL) # We want to add this functionality to the module: # … (as before) T = testMod.buildTuple(6, 0.3, 1) print(T) # A tuple of 6 floats in range 0.3 – 1.0 # I was the last two params to be either # floats or int (any number type) # These should cause an error # T = testMod.buildTuple(3) # Too few parameters # T = testMod.buildTuple(6, 1.4, 1.2) # max before min

11. ADDING A FUNCTION (IMPLEMENTATION) static PyObject * testMod_buildTuple(PyObject * self, PyObject * args) { if (!PyTuple_Check(args) || PyTuple_Size(args) != 3 || !PyLong_Check(PyTuple_GetItem(args, 0)) || !PyNumber_Check(PyTuple_GetItem(args, 1)) || !PyNumber_Check(PyTuple_GetItem(args, 2))) { PyErr_SetString(PyExc_ValueError, "You must pass an int (num elements) and two floats (min & max)"); return NULL; } // Actually extract the parameters from the tuple. int num_elem = PyLong_AsLong(PyTuple_GetItem(args, 0)); PyObject * first_num = PyNumber_Float(PyTuple_GetItem(args, 1)); PyObject * second_num = PyNumber_Float(PyTuple_GetItem(args, 2)); double min_val = PyFloat_AsDouble(first_num); double max_val = PyFloat_AsDouble(second_num); Py_DECREF(first_num); Py_DECREF(second_num); if (min_val > max_val) { PyErr_SetString(PyExc_ValueError, "The min-value must be smaller (or equal to) the max value"); return NULL; }

12. ADDING A FUNCTION (IMPLEMENTATION) continuation of testMod_buildTuple from last slide // Create and fill the tuple. // I want you to figure this part out. You'll likely need the // following functions: PyTuple_New, PyTuple_SetItem and PyFloat_??? [FINISH ME] }

13. ADDING A FUNCTION (IMPLEMENTATION) // Add this line to PyInit_testMod srand(time(NULL)); // Modify the testMod_defintion structure static struct PyModuleDef testMod_definition = { PyModuleDef_HEAD_INIT, "testMod", "My super-awesome first python extension -1, testMod_functions }; // Create this structure (top-level function "table of contents") static PyMethodDef testMod_functions[] = { {"buildTuple", testMod_buildTuple, METH_VARARGS, "Builds a tuple"}, {NULL, NULL, 0, NULL} // Sentinel };

14. PART II: PYTHON/C "CLASSES"

15. PYTHON OOP BASICS (REVIEW) # We're going to do this in C on later slides (and the labs). # Just for comparison here. class Foo(object): """ Class-level docstring """ def __init__(self, name): self.mName = name self.mVal = 0 self.mList = [] def __str__(self): return '["' + self.mName + " : " + str(self.mVal) + "]" def bar(self): self.mVal += 1 self.mList.append(chr(random.randint(97,122))) return self.mVal x = Foo("Bob") print(x.foo()) # 1 print(x) # {"Bob" : 1 : ["q"]}

16. ADDING A CLASS (GOAL) # (Add this to our existing test code) x = testMod.testClass("Bob") y = testMod.testClass("Sue") print(x, y) # ["Bob" : 0], ["Sue" : 0] print(x.foo()) # 1 print(x, y) # ["Bob" : 1], ["Sue" : 0] for i in range(10): y.foo() print(x, y) # ["Bob" : 1], ["Sue" : 10]

17. OVERVIEW Create 4 additional structures: 1.[ testClass ] A C struct defining the C side of that "object" Can contain one or more basic types (floats, ints, etc.) Can contain PyObject's (e.g. for Python strings, lists, etc.) 2.[ testClass_members ] An array of PyMethodDefs structures: one element for each non-special method in the class. 3.[ testClassType ] A PyTypeObject: The definition of the new type (in Python). Includes "slots" for all the special functions (e.g. __init__, __add__, etc) 4.[ testClass_methods ] An array of PyMemeberDef structures: one for each attribute of the class. The rest will be defining the C implementation of all methods. Most will have the signature PyObject* func(PyObject * self, PyObject * args)

18. 1. THE C STRUCT AND 2. MEMBER-DEF typedef struct { PyObject_HEAD PyObject * mName; int mValue; } testClass; static PyMemberDef testClass_members[] = { {"mName", T_OBJECT_EX, offsetof(testClass, mName), 0, "the name of the thingy"}, {"mValue", T_INT, offsetof(testClass, mValue), 0, "the value of the thingy"}, {NULL} // Sentinel };

19. 3. THE PYTYPEOBJECT A big one! Contains slots for all "special" methods (C function pointers) flags, pointers to structures (e.g. our member and method structs) static PyTypeObject testClassType = { PyVarObject_HEAD_INIT(NULL, 0) "testMod.testClass", /* tp_name */ sizeof(testClass), /* tp_basicsize */ 0, /* tp_itemsize */ 0, /* tp_dealloc */ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */

20. 3. THE PYTYPEOBJECT, CONT. 0, /* tp_reserved */ 0, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ 0, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */ "This is the testClass class (within the testMod module)", /* tp_doc. */ 0, /* tp_traverse */ 0, /* tp_clear */

21. 3. THE PYTYPEOBJECT, CONT. 0, /* tp_richcompare */ 0, /* tp_weaklistoffset */ 0, /* tp_iter */ 0, /* tp_iternext */ testClass_methods, /* tp_methods */ testClass_members, /* tp_members */ 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ 0, /* tp_init */ 0, /* tp_alloc */ 0, /* tp_new */ };

22. MODIFICATIONS TO PYINIT_TESTMOD if (PyType_Ready(&testClassType) < 0) return NULL; // Create our module PyObject * mod = PyModule_Create(&testMod_definition); if (mod == NULL) return NULL; Py_INCREF(&testClassType); PyModule_AddObject(mod, "testClass", (PyObject*)&testClassType);

23. 3A. DEFINE TP_NEW AND TP_DEALLOC static PyObject * testClass_new(PyTypeObject * type, PyObject * args, PyObject * kwargs) { testClass * obj = (testClass*)type->tp_alloc(type, 0); if (obj != NULL) { // Initialize mName to an empty string obj->mName = PyUnicode_FromString(""); if (obj->mName == NULL) { Py_DECREF(obj); return NULL; } obj->mValue = 0; } return (PyObject*)obj; } // Insert a pointer to the above function in the tp_new "slot" in testClassType // This effectively re-defines the "__new__" method in python // Insert a pointer to the above function in the tp_dealloc "slot" in testClassType // You'll need to cast the function pointer as a destructor static void testClass_dealloc(testClass * obj) { Py_XDECREF(obj->mName); Py_TYPE(obj)->tp_free((PyObject*)obj); }

24. 3A. DEFINE TP_INIT static int testClass_init(testClass * self, PyObject * args, PyObject * kwargs) { if (!PyTuple_Check(args) || PyTuple_Size(args) != 1 || !PyUnicode_Check(PyTuple_GetItem(args, 0))) { PyErr_SetString(PyExc_TypeError, "You must pass a single string to this method"); return -1;// Error } PyObject * temp_str = PyTuple_GetItem(args, 0); PyObject * old_str = self->mName; Py_INCREF(temp_str); self->mName = temp_str; Py_XDECREF(old_str); self->mValue = 0; return 0;// Success } // Insert a pointer to the above function in the tp_init "slot" in testClassType // (You'll need to cast it as a initproc function pointer // This effectively re-defines the "__init__" method in python

25. 4. DEFINE TESTCLASS_METHODS AND FOO (A NON-SPECIAL METHOD) static PyObject * testClass_foo(PyObject * self, PyObject * args) { testClass * obj = (testClass*)self; obj->mValue++; return PyLong_FromLong(obj->mValue); } static PyMethodDef testClass_methods[] = { {"foo", (PyCFunction)testClass_foo, METH_VARARGS, "Increments mValue and returns it"}, {NULL}// Sentinel };

26. QUESTIONS?