 Hi everybody. Well, I'm going to talk about playing with C Python objects in C Python 3.4. The idea of this talk is to talk about the internal structure of the C Python objects. Well, what do you want in the talk? I want to play with C Python. I want to have 100 plus two, 103. I want to have two equal faults. I want to truncate a tuple. Well, why not? Well, before we can do this crazy stuff, we have to know how is structured the Python objects. Every object... Well, an object in C Python is an instance. It's the same. Every object in Python is a C structure which store data memory in any point of the memory. Every object has a type and only one type. And the type of the object never changed during the lifetime of the object. These have some exceptions in the C Python code, but it's not important for this talk. Every object has an ID. The ID of the object is the address on the memory. And every object in C Python has a reference counter. And when the reference counter reaches zero, the object is free. Well, this is the basic structure of C Python objects. Every C Python object has a list to fields. The OBE of CNT, which is the reference counter, and OBE type, which is a pointer to the object which store the type of each object. Depending on the type of the object, you can have some extra data. Well, I'm going to talk about individual objects in C Python and explain how is the object structure. And we'll play a little with the objects. One object is the simplest object in Python. It has only the reference counter and the OBE type, which is a pointer to the Boolean type. And don't have any extra data. And the non-type, the non-object is a single-ton object for all the C Python interpreter. I'm going to show some examples. And the examples always have this code. I import C types. I define the long size, the insize, and the char size. Okay, for simplicity. I don't show this code in the next slides. And I have some slides doing crazy stuff to the C Python objects. And I call these sections very bad things. Okay? I call these sections very bad things because you don't have to do this. You avoid to do the things I do in these slides. Well, I talk about known. Known have two fields, two long fields, which are eight bytes. These are 16 bytes for known, for nothing. This sounds like a memory leak. Well, we know that when the reference counter reaches zero, the memory is freed. Then we know that the idea of an object is the address in memory. And with this information and some help of C types, I go into free the known object because it's a memory leak. Well, the first thing, I get the idea of known and read the address, read the data in this address position. It's a long, which is the reference counter of known. I set this value to zero. And I free the known object. Well, with some collateral damage like the C Python interpreter. Well, it's important. I free the memory leak. The end object. The end object have four fields. The reference counter and the OV type have an OV size, which is the size of the OV digit array. The OV digit array is an array of integers. And the value of the end object is calculated with the formula at the bottom of the list. I think it's more clear in the example. I can have the seven value in an integer object. And this is stored as OV size of one and OV digit of seven. And we can have 1024 qubit and the OV size is two. And the first OV digit position have a zero. And the second OV digit position have a one. This structure allows to have arbitrary long integers in Python. Well, I can access to the structure of end with the type. I set the X variable to 100. I get the OV size and OV size is one. I get the OV digit and OV digit is 100. With 1024 qubit, I get the OV size and it's two. And the first OV digit is zero. And the second OV digit is one. Well, let's play a little. I can set the variable X to 1000. I get the OV digit of variable X. I set this value to 101. And, well, integers are immutables. Well, immutable. With the types, it's false. But, okay, I get X, X now is 100, 1001. And 1000 is 1000. Okay. Well, 1000 is too much. Let me be a little bit stinky. 100 is okay, okay? I set X variable to 100. Now I change the value of 100, the OV digit of 100. And now X is 101. Okay. But 100 now is 101. What? Wait, wait, wait, wait. 100 plus two is 103. Well, this has an explanation. The CPython interpreter cached the value from minute five to 256. And these objects, these three objects are reused by all the CPython interpreter. Then when you use an integer in this range, you are using this cached integer. Use the same instance for all the values using this integer. And this is the reason. It's not a book. The bull object. The bull object is an integer. The true value is an integer with OV size one and OV digit one. And the false variable have the OV size zero and OV digit zero. There are singleton variables, too. You like the known object. And the OV type points to the Boolean class, to the Boolean type. I can access to the OV size on OV digit of true and false, and it's one and one and zero and zero. Well, if I get the OV digit of, the OV size of true and set it to zero, I get the OV digit of true and set it to zero. And now true is equal to false. True. Nice. Another thing. The type, the Boolean type, is too similar to the int type that I have to play with this. Then I get the OV type of Boolean. This is this address, which is the same of the idea of the class Boo. I get the type address of the field, I get the field OV type of the true variable and set it to the class int. And now true have the representation of an integer. Cool. The bytes, the bytes objects. The bytes objects have this structure with reference counter, OV type. OV size is the number of bytes. The OV hash is the hash of the OV, what do, is the hash of the bytes contained in the bytes structure. And OV as well is on the right of bytes with zero binary zero at the end, like string c and c. For example, if I have the A byte, I have an OV size of one and OV as well of one position A and two positions, A and binary zero, A, B have an OV size of two and three positions A, B and binary zero. I can access to JEP. I get the OV size and the three. I get the hash and match with the OV hash value. I get the first byte of OV as well and I get YEP and the binary zero. The top of the object. The top of the object has a reference counter and OV type and the OV size, which is the number of elements of the top of the OV item, which is an array of pointers to Python objects. For example, I can have the true false table which have the OV size of two and an array of two pointers, one pointer to a true variable and one pointer to the false variable. I can access to a table. I get the OV size and S2. I get the first position and it's a pointer, which and the second position is another pointer and this pointer is matched with the true and false variables. Well, a table. Another immutable structure. A list with immutability. Well, immutability. I set one, two, three to X variable. I change the OV size to two and now I have magically converted a three elements table in a two elements table plus a memory leak. Magic. Well, the list object. The list object has a reference counter, an OV type and OV size, which is the number of elements in the list. The OV item is a pointer to a structure in memory, which is an array of pointers to Python objects. We'll see better in the example. And the allocated is the quantity of memory reserved for these objects, for these pointers. In this example, I can have true and false lists, which have OV size and OV item is a pointer to a block of memory, which have the address of true and the address of false. An allocated is equals to, in this case, but the allocated memory can not match with the OV size. We can have bigger allocated memory blocks and we only use some for the elements. Well, it depends on the way you have created the list. Well, to access a list, I get the OV size is free, I get the OV item pointer and this address in the memory. I get the first position of this address is a pointer to a Python object and this pointer is one. Well, I can get the second pointer and this pointer will be true, will be true. Another very good thing. I set the variable X to an element list and the variable Y to a full element list. I get the OV item pointer of Y, I get the OV item pointer of X and I set the Y OV item pointer to the X OV item pointer. Then, if I get Y, I get the first four positions of Y of X. If I modify X, I can see these modifications in Y. This is not a great feature for using production, okay? It's a crazy thing that don't use this code, please. Well, the big object. The big object is the more complex object that I will show in this talk. It's a complex structure and really nice. There are a pep explaining how it works and the pep 4.1.2 talk about the split tables and community tables structure of the Python dictionaries. Well, every dict in Python have a reference counter OV type, MA used, which is the number of keys used by the dict. The MA keys point to a data structure and MA values store. It's a pointer to the values in some types of dictionaries. I think we'll be better to see it in the example. The decay reference counter is a reference counter for this block, this set of keys. Decay size, okay. Decay size is the total size of the hash map Decay lookup is a slot for function to look up the hash in the entries. Decay useable is the left space on the dictionary useable before the dictionary needs a resize. Decay entries is an array of key and key is a key. Decay entry structure is a hash of the key, the ME key is a pointer to the key and ME value is a pointer to the value. Let's see an example. With a dictionary, 1, 2, 3, 4, I have two keys, then the MA used is two. The MA values is zero because this is an example of convenient tables dictionary. This means the values are stored in the key entries. The ME key points to a data structure. The data structure has a decay size of eight because we have eight positions to store data or store keys. Decay useable has a value of three. This means if I store three values more or three keys more, the dictionary will be resized and the dictionary keys, the entry keys will have an ME key of, will be the address of one, the ME value will be the address of two, the ME key will be the address of three and four. The hash, the hash function, the default hash function in CPython for integers is the same value of the integer. Then the hash, in this case, is easy to know. The splitter tables structure is used mainly for store the value of attribute of objects and, well, it's for don't repeat every time the keys and the values in the dictionary object of the objects in Python. The obi-ri-ri-cian-t, well, I have only one attribute, then the ME used is one. The ME values have one value, in this case, is the address of one. The decay size is A, the decay usable is three and the ME key will be the pointer to the A key and the ME hash will be the hash of the A and ME value in this case is zero because it's binary zero because we are using splitter tables in this case. To access a dictionary, it's a little bit complex, but, well, I get the keys, the pointer to the keys, I get the KN31, the KN37 in this case and, well, I get the key, the value. All these examples will be published in Github and the slide will be published in speaker deck. The key and the value, I get the key and the value, the key and the value of seven. I get the OBD of the first key one, the OBD of key, of bar one, which is three, the OBD of key seven and the OBD of bar seven. Well, extra valve. This is, I don't explain the code enough for understanding this example, but, well, I think it's funny. I will change the add magic method of integer globally. I import the types. I define a callback function in Ctypes, which gets two Python objects and returns one Python object and returns always 42. Now, I get the address of the pointer to the C function that wraps this Python function. I get the integer class data structure. I find the add number address, which is a pointer to the data structure with all the slots of functions, of the functions that implement the number behavior in integers. I get the first position of this data structure, which is the slot for add function. I put my function. I increment the reference counter of 42 for not break the interpreter. And I print one plus one, and I get 42. Well, reference. The main reference of this talk is the Python code. The C Python object is not especially complex to understand with Fama sections like Unicode. The Python objects are defined in the include directory and the object directory. And you don't have to see the rest of Python to understand the C Python objects. The Ctypes documentation is really interesting for play with the C Python data structure. The C API of Python is really interesting too. If you want to know more about the C Python dictionary, you can read the PEP 412. And if you want to see the access code of the examples of the slides in my Github, and the very bad things, examples in my Github too. Well, the conclusions. I don't write C code normally. I'm not a core developer of Python. I don't have a big knowledge of C Python interpreter. But I can play with the C Python objects. C Python objects are simple and not complex. And can be funny. And of course, if I can... Don't be scared about the C Python source code. If I can read it, you can read it. Sure. Questions? Questions? Yeah, this. Okay, first sensible talk. Very interesting. And my question is, what are the possible practical usages of this, except for entertaining the audience? Thanks. I hope for now. This is not for practical use. I use Ctypes for play with the C Python objects, but I don't find any practical use of this knowledge. The talk is not for showing something to use in your normal days. It's for better understanding of the C Python, how C Python works. I hope not to see this code in any production environment or something like that, because it's crazy stuff. More questions? Could this type of code not cause any security problems if you have any bad ideas and you want to access certain objects that should not be accessible normally? Can you repeat the English? It's not really good. Can I use this in a bad way and create a problem with security? I guess private functions, private variables. Private functions in Python are not private. Then the security is not a problem. We have, well, the problem is not the security, the security is broken anyway. Yes, you can access to all the process memory through PROC directory, and then the security is broken anyway. I find it interesting for access to information that you don't know or you don't have to know, but if you share binary code, you can access with these techniques to this information, but there are a lot of other techniques for access to this information. I think it's not a better tool for access to private information. So if you make all these modifications, it would only affect the current process, right? So if you open another interpreter, Python would behave normally? Of course. These changes are done in the memory of the process. You don't have access to modify other Python interpreters. You can create modules that, when are in port, do things like that, but it's a bad idea. Or a good idea for half a funny day with a co-worker. Do you have more questions from the audience? If no, let's thank the speaker again. Thanks.