1. Python C API overview References: http://docs.python.org/3.2/extending/index.html#extending-index http://docs.python.org/3.2/c-api/

2. Two(or three) types Extending – Write a dll (.pyd), which exports an initialization function a list of "things" in the module – classes – functions – variables – You can import this just like any (pure) python module – Use the c types / functions in your program import mymodule mymodule.fun(4.1) # A c-function

3. Two (or three) types, cont. Embedding – Create a python interpreter inside a C program – Can execute python code Need to convert to/from python types Hybrid – Both! – In this case, though, you don't actually create a dll (but you do everything you did in the dll) – This is what we'll use for our scripting.

4. Naming convention PyXXX_YYY – (Python) class XXX, functionYYY – e.g. PyTuple_Size(…) Py_ZZZ – Global function ZZZ – e.g. Py_Compile(…)

5. PyObject's Literally everything in Python is a PyObject* – Int ref-count (more on this shortly) – A type structure – Any other data e.g. for tuples, they have a length attribute. The python interpreter owns all objects – The user just has references to them – The first time it is referenced, python malloc's it. – When the refcnt reaches 0, it eventually gets free'd – In (normal) python this is done for us… – …but in the C API we have to manage it.

6. RefCounting Take a simple example def foo(a): return a ** 2 x = 4 y = [foo(x), 3.2] z = y Line 1 – 2 creates a new function object. Line 3 creates a new int x. Line 4 calls foo. o a and x refer to the same thing (refcnt is 2) o Creates another float and returns it o a goes out of scope – the refcnt on the int is now 1 o The 16.0 and 3.2 are part of a new list object (refcnt = 1) Line 5 makes a second reference to the list (Refcnt = 2)

7. RefCounting, cont. …In the C API, you have to manually do it. – Memory Leaks (if you forget to decrement a ref-count) – Invalid memory access (if you forget to increment, and then the object is destroyed) Three functions Py_INCREF(PyObject*); Py_DECREF(PyObject*); Py_XDECREF(PyObject*); // Like previous, but has a NULL- check

8. RefCounting, cont. Look in the C/API documentation – http://docs.python.org/py3k/c-api/ – If you see New Reference e.g. PyLong_FromLong(long v) You own a ref-count to it. Make sure to call Py_DECREF when done. – If you see Borrowed Reference e.g. PyList_GetItem(PyObject * list, Py_ssize_t index) Someone else holds a reference Probably safest to: – call Py_INCREF – Use it – call Py_DECREF

9. Converting between types Integers – C => Python PyObject * PyLong_FromLong(int i); A new reference – make sure you DECREF it! – Python => C Int PyLong_Check(PyObject*); int PyLong_AsLong(PyObject*); float PyLong_AsDouble(PyObject*); …

10. Converting between types, cont. Floats – C => Python PyObject * PyFloat_FromDouble(double d); A new reference – make sure you DECREF it! – Python => C double PyFloat_AsDouble(PyObject*);

11. Converting between types, cont. Strings – A bit more complicated because python 3.x uses unicode internally, not ANSI strings. – C => Python PyObject * PyUnicode_FromString(char * s); A new reference – make sure you DECREF it! – Python => C char temps[256]; // PObj is the object (in python) we want to convert PyObject * pBytesObj = PyUnicode_AsUTF8String(pObj); strcpy(temps, PyBytes_AsString(pBytesObj)); Py_DECREF(pBytesObj);

12. Creating a C extension (pyd) Goal: – A new class (written in C) – A new function (written in C) – A variable (written in C) – Access all 3 in python import myCMod C = myCMod.myClass("Bob") print(C.name) # Bob print(C.timestamp) # 0 C.incTimestamp() print(C.timestamp) # 1 print(myCMod.fact(5)) # 120 print(myCMod.aVariable) # Surprise!

13. Making an extension (pyd) 0. Set up a project – A Win32 console dll (empty) – Put all your code in a single.cpp file – Change output to xyz.pyd (actually a.dll) – Only do a release build (debug can only be run by a debug build of python) – Include includes / libs Reference: http://docs.python.org/3.2/extending/index.html#extending-index

14. Making an extension (pyd) 1.Create a C-type ("guts" of a python object) typedef struct { PyObject_HEAD /* Type-specific fields go here. */ PyObject * name; int timestamp; } myClass;

15. Making an extension (pyd) 2.Create methods for the new type // An "overload" of the allocation func static void myClass_new(PyTypeObject * type, PyObject * args, PyObject * kwds) { myClass * self; self = (myClass*)type->tp_alloc(type, 0); // Note: init is called later. This is // for code that needs to happen before that. self->name = NULL; } // An "overload" of the delete func static void myClass_dealloc(myClass * c) { Py_XDECREF(c->name); Py_TYPE(c)->tp_free((PyObject*)c); }

16. Making an extension (pyd) // An "overload" of the init method static int myClass_init(myClass * self, PyObject * args, PyObject * kwds) { PyObject * newName; // Get a single objectfrom the args if (!PyArg_ParseTuple(args, "O", &newName)) return -1; // ERROR – nothing in args. Py_INCREF(newName); self->name = newName; self->timestamp = 0; return 0; // Everything OK } // A new method (not an overload) static void incTimeStamp(myClass * self) { self->timestamp++; }

17. Making an extension (pyd) 3.Create an attribute and method structure for the new type // Define the attributes here static PyMemberDef myClass_members[] = { {"name", T_OBJECT_EX, offsetof(myClass, name), 0, "our name"}, {"timeStamp", T_OBJECT_EX, offsetof(myClass, timeStamp), 0, "timestamp"}, {NULL} /* Sentinel */ }; // Define the non-standard methods here static PyMethodDef myClass_methods[] = { {"incTimestamp", (PyCFunction)incTimestamp, METH_NOARGS, "Increments our timestamp" }, {NULL} /* Sentinel */ };

18. Making an extension (pyd) 4.Create a "dictionary" of functions and attributes for the new type static PyTypeObject MyClassType = { PyVarObject_HEAD_INIT(NULL, 0) "myClass", /* tp_name */ sizeof(myClass), /* tp_basicsize */ 0, /* tp_itemsize */ (destructor)my_dealloc, /* tp_dealloc */ 0, /* tp_print */ 0, /* tp_getattr */ 0, /* tp_setattr */ 0, /* tp_reserved */ 0, /* tp_repr */ 0, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ 0, /* tp_hash */ 0, /* tp_call */ 0, /* tp_str */ 0, /* tp_getattro */ 0, /* tp_setattro */

19. Making an extension (pyd) 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */ "The coolest object ever", /* tp_doc */ 0, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ 0, /* tp_weaklistoffset */ 0, /* tp_iter */ 0, /* tp_iternext */ myClass_methods, /* tp_methods */ myClass_members, /* tp_members */ 0, /* tp_getset */ 0, /* tp_base */ 0, /* tp_dict */ 0, /* tp_descr_get */ 0, /* tp_descr_set */ 0, /* tp_dictoffset */ (initproc)my_init, /* tp_init */ 0, /* tp_alloc */ Noddy_new, /* tp_new */ };

20. Making an Extension 5.Create a C function in the module (factorial) – I'll let you experiment with this 6.Create a C variable in the module – I'll let you experiment with this too…

21. Making an Extension 7.Make a structure containing the module static PyModuleDef my_module = { PyModuleDef_HEAD_INIT, "myCMod", "my awesome module", -1, // No extra memory needed ???, // module funcs. A pointer to a PyMethodDef structure NULL, NULL, NULL, NULL };

22. Making an Extension 8.Make the init function (exported in the dll). Note: make sure it's the same name as the dll so it's auto-imported. PyMODINIT_FUNC PyInit_mycmod(void) { PyObject* m; // Ideally we'd do this in step 4. Some compilers // complain, though, so more portable to do it here. MyClassType.tp_new = PyType_GenericNew; if (PyType_Ready(&MyClassType) < 0) return NULL; m = PyModule_Create(&my_module); if (m == NULL) return NULL; Py_INCREF(&MyClassType); PyModule_AddObject(m, "myCMod", (PyObject *)&MyClassType); return m; }