diff --git a/docs/C-API/introduction.md b/docs/C-API/introduction.md index ba193b3b..faf16380 100644 --- a/docs/C-API/introduction.md +++ b/docs/C-API/introduction.md @@ -4,36 +4,4 @@ icon: dot order: 10 --- -### What C-API is for - -If your target platform does not support C++17. You can compile pkpy into a static library and use its exported C-APIs. - -Our C-APIs take a lot of inspiration from the lua C-APIs. -Methods return a `bool` indicating if the operation succeeded or not. -Special thanks for [@koltenpearson](https://github.com/koltenpearson)'s contribution. - -!!! -C-APIs are always stable and backward compatible. -!!! - -### Basic functions - -+ `pkpy_vm* pkpy_new_vm(bool enable_os)` - - Wraps `new VM(enable_os)` in C++. - -+ `void pkpy_delete_vm(pkpy_vm*)` - - Wraps `delete vm` in C++. - -+ `bool pkpy_exec(pkpy_vm*, const char* source)` - - Wraps `vm->exec`. Execute a string of source code. - -+ `bool pkpy_exec_2(pkpy_vm*, const char* source, const char* filename, int mode, const char* module)` - - Wraps `vm->exec_2`. Execute a string of source code with more options. - -+ `void pkpy_set_main_argv(pkpy_vm*, int argc, char** argv)` - - Wraps `vm->set_main_argv`. Set the `sys.argv` before executing scripts. +TBA \ No newline at end of file diff --git a/docs/C-API/stack.md b/docs/C-API/stack.md deleted file mode 100644 index 793304b3..00000000 --- a/docs/C-API/stack.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -title: Stack Manipulation -icon: dot -order: 8 ---- - -### Basic manipulation - -+ `bool pkpy_dup(pkpy_vm*, int)` - - Duplicate the value at the given index. - -+ `bool pkpy_pop(pkpy_vm*, int)` - - Pop `n` values from the stack. - -+ `bool pkpy_pop_top(pkpy_vm*)` - - Pop the top value from the stack. - -+ `bool pkpy_dup_top(pkpy_vm*)` - - Duplicate the top value on the stack. - -+ `bool pkpy_rot_two(pkpy_vm*)` - - Swap the top two values on the stack. - -+ `int pkpy_stack_size(pkpy_vm*)` - - Get the element count of the stack. - - -### Basic push, check and convert - -+ `pkpy_push_xxx` pushes a value onto the stack. -+ `pkpy_is_xxx` checks if the value at the given index is of the given type. -+ `pkpy_to_xxx` converts the value at the given index to the given type. - -Stack index is 0-based instead of 1-based. And it can be negative, which means the index is counted from the top of the stack. - -```c -// int -PK_EXPORT bool pkpy_push_int(pkpy_vm*, int); -PK_EXPORT bool pkpy_is_int(pkpy_vm*, int i); -PK_EXPORT bool pkpy_to_int(pkpy_vm*, int i, int* out); - -// float -PK_EXPORT bool pkpy_push_float(pkpy_vm*, double); -PK_EXPORT bool pkpy_is_float(pkpy_vm*, int i); -PK_EXPORT bool pkpy_to_float(pkpy_vm*, int i, double* out); - -// bool -PK_EXPORT bool pkpy_push_bool(pkpy_vm*, bool); -PK_EXPORT bool pkpy_is_bool(pkpy_vm*, int i); -PK_EXPORT bool pkpy_to_bool(pkpy_vm*, int i, bool* out); - -// string -PK_EXPORT bool pkpy_push_string(pkpy_vm*, pkpy_CString); -PK_EXPORT bool pkpy_is_string(pkpy_vm*, int i); -PK_EXPORT bool pkpy_to_string(pkpy_vm*, int i, pkpy_CString* out); - -// void_p -PK_EXPORT bool pkpy_push_voidp(pkpy_vm*, void*); -PK_EXPORT bool pkpy_is_voidp(pkpy_vm*, int i); -PK_EXPORT bool pkpy_to_voidp(pkpy_vm*, int i, void** out); - -// none -PK_EXPORT bool pkpy_push_none(pkpy_vm*); -PK_EXPORT bool pkpy_is_none(pkpy_vm*, int i); -``` - -### Special push - -+ `pkpy_push_null(pkpy_vm*)` - - Push a `PY_NULL` onto the stack. It is used for `pkpy_vectorcall`. - -+ `pkpy_push_function(pkpy_vm*, const char* sig, pkpy_CFunction f)` - - Push a function onto the stack. `sig` is the function signature, e.g. `add(a: int, b: int) -> int`. `f` is the function pointer. - -+ `pkpy_push_module(pkpy_vm*, const char* name)` - - Push a new module onto the stack. `name` is the module name. This is not `import`. It creates a new module object. - -### Variable access - -+ `bool pkpy_getattr(pkpy_vm*, pkpy_CName name)` - - Push `b.` onto the stack. Return false if the attribute is not found. - - ``` - [b] -> [b.] - ``` - -+ `bool pkpy_setattr(pkpy_vm*, pkpy_CName name)` - - Set `b.` to the value at the top of the stack. - First push the value, then push `b`. - - ``` - [value, b] -> [] - ``` - -+ `bool pkpy_getglobal(pkpy_vm*, pkpy_CName name)` - - Push a global/builtin variable onto the stack. Return false if the variable is not found. - - ``` - [] -> [value] - ``` - -+ `bool pkpy_setglobal(pkpy_vm*, pkpy_CName name)` - - Set a global variable to the value at the top of the stack. - - ``` - [value] -> [] - ``` - -+ `bool pkpy_eval(pkpy_vm*, const char* source)` - - Evaluate a string and push the result onto the stack. - - ``` - [] -> [result] - ``` - -+ `bool pkpy_unpack_sequence(pkpy_vm*, int size)` - - Unpack a sequence at the top of the stack. `size` is the element count of the sequence. - - ``` - [a] -> [a[0], a[1], ..., a[size - 1]] - ``` - -+ `bool pkpy_get_unbound_method(pkpy_vm*, pkpy_CName name)` - - It is used for method call. - Get an unbound method from the object at the top of the stack. `name` is the method name. - Also push the object as self. - - ``` - [obj] -> [obj. self] - ``` -+ `bool pkpy_py_repr(pkpy_vm*)` - - Get the repr of the value at the top of the stack. - - ``` - [value] -> [repr(value)] - ``` - -+ `bool pkpy_py_str(pkpy_vm*)` - - Get the str of the value at the top of the stack. - - ``` - [value] -> [str(value)] - ``` - -+ `bool pkpy_py_import(pkpy_vm*, const char* name)` - - Import a module and push it onto the stack. - - ``` - [] -> [module] - ``` diff --git a/docs/modules/c.md b/docs/modules/c.md deleted file mode 100644 index 4f1909df..00000000 --- a/docs/modules/c.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -icon: package -label: c ---- - -Interop with pointers and C structs. - -https://github.com/pocketpy/pocketpy/blob/main/include/typings/c.pyi \ No newline at end of file diff --git a/docs/modules/colorsys.md b/docs/modules/colorsys.md deleted file mode 100644 index 12b74be1..00000000 --- a/docs/modules/colorsys.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -icon: package -label: colorsys ---- - -The same as the standard library module `colorsys` in python 3.11. - -https://github.com/python/cpython/blob/3.11/Lib/colorsys.py \ No newline at end of file diff --git a/docs/modules/csv.md b/docs/modules/csv.md deleted file mode 100644 index 47679457..00000000 --- a/docs/modules/csv.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -icon: package -label: csv ---- - -### `csv.reader(csvfile: list[str]) -> list[list]` - -Parse a CSV file into a list of lists. - -### `csv.DictReader(csvfile: list[str]) -> list[dict]` - -Parse a CSV file into a list of dictionaries. - -## Example - -```python -import csv - -data = """a,b,c -1,2,3 -""" - -print(csv.reader(data.splitlines())) -# [ -# ['a', 'b', 'c'], -# ['1', '2', '3'] -# ] - -print(csv.DictReader(data.splitlines())) -# [ -# {'a': '1', 'b': '2', 'c': '3'} -# ] -``` \ No newline at end of file diff --git a/docs/quick-start/installation.md b/docs/quick-start.md similarity index 67% rename from docs/quick-start/installation.md rename to docs/quick-start.md index 7c7400b5..4d039039 100644 --- a/docs/quick-start/installation.md +++ b/docs/quick-start.md @@ -1,7 +1,7 @@ --- -icon: dot -label: 'Installation' -order: 100 +icon: rocket +order: 20 +label: Quick Start --- You have two options to integrate pkpy into your project. @@ -126,55 +126,3 @@ __ERROR: return 1; } ``` - -### Overview - -pkpy's C++ interfaces are organized in an object-oriented way. -All classes are located in `pkpy` namespace. - -The most important class is the `VM` class. A `VM` instance is a python virtual machine which holds all necessary runtime states, including callstack, modules, variables, etc. - -A process can have multiple `VM` instances. Each `VM` instance is independent from each other. - -!!! -Always use C++ `new` operator to create a `VM` instance. -DO NOT declare it on the stack. It may cause stack overflow. -!!! - -```cpp -VM* vm = new VM(); -``` - -The constructor can take 1 extra parameters. - -#### `VM(bool enable_os=true)` - -+ `enable_os`, whether to enable OS-related features or not. This setting controls the availability of privileged modules such os `io` and `os` as well as builtin function `open`. **It is designed for sandboxing.** - -When you are done with the `VM` instance, use `delete` operator to dispose it. - -```cpp -delete vm; -``` - -### Hook standard buffer - -By default, pkpy outputs all messages and errors to `stdout` and `stderr`. -You can redirect them to your own buffer by setting `vm->_stdout` and `vm->_stderr`. - -These two fields are C function pointers with the following signature: - -```cpp -void(*)(const char*, int); -``` - -Or you can override these two virtual functions: -```cpp - virtual void stdout_write(const Str& s){ - _stdout(s.data, s.size); - } - - virtual void stderr_write(const Str& s){ - _stderr(s.data, s.size); - } -``` \ No newline at end of file diff --git a/docs/quick-start/attr.md b/docs/quick-start/attr.md deleted file mode 100644 index eccac8c7..00000000 --- a/docs/quick-start/attr.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -icon: dot -label: 'Access Attributes' -order: 80 ---- - -### Direct access - -Some python objects have an instance dict, a.k.a, `__dict__` in cpython. -You can use `obj->attr()` to manipulate the instance dict of an object. - -```cpp -VM* vm = new VM(); - -// get the `builtin` module -PyVar builtins = vm->builtins; -// get `dict` type -PyVar dict = builtins->attr("dict"); -// set `pi = 3.14` -builtins->attr().set("pi", py_var(vm, 3.14)); -``` - -However, you cannot call `attr` on an object which does not have an instance dict. -For example, the `int` object. - -```cpp -// create a `int` object -PyVar obj = py_var(vm, 1); -// THIS IS WRONG!! WILL LEAD TO A SEGFAULT!! -PyVar add = obj->attr("__add__"); -``` - -To determine whether an object has instance dict or not, you can use this snippet. - -```cpp -// 1. call `is_tagged` to check the object supports `->` operator -// 2. call `is_attr_valid` to check the existence of instance dict -PyVar obj = py_var(vm, 1); -bool ok = !is_tagged(obj) && obj->is_attr_valid(); // false -``` - -### General access - -As you can see, direct access does not take care of derived attributes or methods. -In most cases, what you need is `getattr` and `setattr`. -These two methods handle all possible cases. - -#### `PyVar getattr(PyVar obj, StrName name, bool throw_err=true)` - -This method is equivalent to `getattr` in python. -If the attribute is not found, it will return `nullptr` -or throw an `AttributeError` depending on the value of `throw_err`. - -```cpp -// create a `int` object -PyVar obj = py_var(vm, 1); - -// get its `__add__` method, which is a `bound_method` object -PyVar add = vm->getattr(obj, "__add__"); - -// call it (equivalent to `1 + 2`) -PyVar ret = vm->call(add, py_var(vm, 2);); - -// get the result -int result = py_cast(vm, ret); -std::cout << result << std::endl; // 3 -``` - -#### `void setattr(PyVar, StrName, PyVar)` - -This method is equivalent to `setattr` in python. -It raises `TypeError` if the object does not support attribute assignment. diff --git a/docs/quick-start/call.md b/docs/quick-start/call.md deleted file mode 100644 index 5ce38393..00000000 --- a/docs/quick-start/call.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -icon: dot -label: 'Call Python Function' -order: 70 ---- - -pkpy uses a variant of the [Vectorcall](https://peps.python.org/pep-0590/) protocol (PEP 590). - -You can use `call` to invoke any python callable object, -including functions, methods, classes, etc. -For methods, `call_method` can be used. - -+ `PyVar call(PyVar obj, ...)` -+ `PyVar call_method(PyVar obj, StrName name, ...)` - -### Example - -Let's create a `dict` object and set a key-value pair, -which equals to the following python snippet. - -```python -obj = {} # declare a `dict` -obj["a"] = 5 # set a key-value pair -print(obj["a"]) # print the value -``` - -First, create an empty dict object, - -```cpp -PyVar tp = vm->builtins->attr("dict"); -PyVar obj = vm->call(tp); // this is a `dict` -``` - -And set a key-value pair, - -```cpp -PyVar _0 = py_var(vm, "a"); -PyVar _1 = py_var(vm, 5); -vm->call_method(obj, "__setitem__", _0, _1); -``` - -And get the value, - -```cpp -PyVar ret = vm->call_method(obj, "__getitem__", _0); -std::cout << py_cast(vm, i64); -``` - -If you want to call with dynamic number of arguments, -you should use `vm->vectorcall`. This is a low-level, stack-based API. - -1. First push the callable object to the stack. -2. Push the `self` object to the stack. If there is no `self`, push `PY_NULL`. -3. Push the arguments to the stack. -4. Call `vm->vectorcall` with the number of arguments. - -```cpp -PyVar f_sum = vm->builtins->attr("sum"); - -List args(N); // a list of N arguments - -vm->s_data.push_back(f_sum); -vm->s_data.push_back(PY_NULL); // self - -for(PyVar arg : args) { - vm->s_data.push_back(arg); -} - -PyVar ret = vm->vectorcall(args.size()); -``` - diff --git a/docs/quick-start/config.md b/docs/quick-start/config.md deleted file mode 100644 index f7c34d07..00000000 --- a/docs/quick-start/config.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -icon: dot -label: 'Advanced Config' -order: -2 ---- - -### Enable os-related features - -If you want to enable os-related features, you can do this before including `pocketpy.h`. - -```cpp -#define PK_ENABLE_OS 1 -#include -``` - -### Working with multiple threads - -pkpy does not support multi-threading. But you can create multiple `VM` instances and run them in different threads. -You can do the following to ensure thread safety for `VM` instances: - -```cpp -#define PK_ENABLE_THREAD 1 -#include -``` - -### Full config - -You can take a look at `include/pocketpy/config.h` to see all the available configuration macros. - - diff --git a/docs/quick-start/exec.md b/docs/quick-start/exec.md deleted file mode 100644 index cbc88b20..00000000 --- a/docs/quick-start/exec.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -icon: dot -label: 'Execute Python Code' -order: 93 ---- - -### Simple execution - -Once you have a `VM` instance, you can execute python code by calling `exec` method. - -#### `PyVar exec(Str source, Str filename, CompileMode mode, PyVar _module=nullptr)` - -+ `source`, the python source code to be executed -+ `filename`, the filename of the source code. This is used for error reporting -+ `mode`, the compile mode. See below for details -+ `module`, the module where the code will be executed. If `nullptr`, the code will be executed in the `__main__` module - -`exec` handles possible exceptions and returns a `PyVar`. -If the execution is not successful, e.g. a syntax error or a runtime exception, -the return value will be `nullptr`. - -There are also overloaded versions of `exec` and `eval`, which is useful for simple execution: -+ `PyVar exec(Str source)` -+ `PyVar eval(Str source)` - -### Compile mode - -The `mode` parameter controls how the source code is compiled. There are 5 possible values: -+ `EXEC_MODE`, this is the default mode. Just do normal execution. -+ `EVAL_MODE`, this mode is used for evaluating a single expression. The `source` should be a single expression. It cannot contain any statements. -+ `REPL_MODE`, this mode is used for REPL. It is similar to `EXEC_MODE`, but generates `PRINT_EXPR` opcode when necessary. -+ `CELL_MODE`, this mode is designed for Jupyter like execution. It is similar to `EXEC_MODE`, but generates `PRINT_EXPR` opcode when necessary. -+ `JSON_MODE`, this mode is used for JSON parsing. It is similar to `EVAL_MODE`, but uses a lexing rule designed for JSON. - - -### Fine-grained execution - -In some cases, you may want to execute python code in a more fine-grained way. -These two methods are provided for this purpose: - -+ `CodeObject_ compile(Str source, Str filename, CompileMode mode, bool unknown_global_scope)` -+ `PyVar _exec(CodeObject_ co, PyVar _module)` - -1. `compile` compiles the source code into a `CodeObject_` instance. Leave `unknown_global_scope` to `false` if you don't know what it means. -2. `_exec` executes the `CodeObject_` instance. - -!!! -`_exec` does not handle exceptions, you need to use `try..catch` manually. -!!! - -```cpp -try{ - CodeObject_ code = vm->compile("a[0]", "main.py", EXEC_MODE, false); - vm->_exec(code, vm->_main); -}catch(TopLevelException e){ - // use e.summary() to get a summary of the exception - std::cerr << e.summary() << std::endl; -} -``` diff --git a/docs/quick-start/index.yml b/docs/quick-start/index.yml deleted file mode 100644 index 6a0facfc..00000000 --- a/docs/quick-start/index.yml +++ /dev/null @@ -1,3 +0,0 @@ -icon: rocket -order: 20 -label: Quick Start diff --git a/docs/quick-start/interop.md b/docs/quick-start/interop.md deleted file mode 100644 index 9192e64f..00000000 --- a/docs/quick-start/interop.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -icon: dot -label: 'Interop with PyVar' -order: 90 ---- - -In pkpy, any python object is represented by a `PyVar`. - - -### Create `PyVar` from C type - -A set of overloaded function `PyVar py_var(VM* vm, ...)` were implemented to -create a `PyVar` from a supported C type. - -Assume we have a `VM* vm` instance. -You can create a python `int` object from a C `i64` type: - -```cpp -i64 i = 2; -PyVar obj = py_var(vm, i); -``` - -Each python type has a corresponding C type, for example, `int` in python is `i64` in C. -python's `list` corresponds to `List`, `str` corresponds to `Str`, etc. -For strings, we have defined -a set of overloaded version including `const char*`, `std::string`, `std::string_view`, `Str`, etc. - -```cpp -PyVar obj = py_var(vm, "abc"); // create a python str object -``` - -A more complex example is to create a python `list`. -In the following code, we create a `list` equals to `[0, 1, 2, 3]`. - -```cpp -List list; -for (i64 i = 0; i < 4; i++) { - list.push_back(py_var(vm, i)); -} - -obj = py_var(vm, std::move(list)); // create a python list object -``` - -Please note that `std::move` is used here to avoid unnecessary copy. -Most types have both a rvalue and a lvalue version of `py_var` function. - -### Access internal C type of `PyVar` - -A set of template function `T py_cast(VM* vm, PyVar obj)` were implemented. - -```cpp -i64 i = 2; -PyVar obj = py_var(vm, i); - -// cast a PyVar to C i64 -i64 j = py_cast(vm, obj); -``` - -The `py_cast` function will check the type of `obj` before casting. -If the type is not matched, a `TypeError` will be thrown. - -However, this type check has a cost. If you are sure about the type of `obj`, -you can use the underscore version `_py_cast` to skip the type check. - -```cpp -// cast a PyVar to C i64 (unsafe but faster) -i64 j = _py_cast(vm, obj); -``` - -For complex objects like `list`, we can use reference cast to avoid unnecessary copy. - -```cpp -PyVar obj = py_var(vm, List()); -// reference cast (no copy) -List& list = py_cast(vm, obj); -``` - -### Check type of `PyVar` - -Each `PyVar` has a `Type type` field to indicate its type. -`Type` is just an integer which is the global index in `vm->_all_types`. - -`VM` class has a set of predefined `Type` constants for quick access. -They are prefixed by `tp_`. For example, `tp_object`(object), -`tp_int`(int), `tp_str`(str), `tp_list`(list), etc. - -Types are divided into **tagged type** and **non-tagged type**. -+ small `int` objects are tagged. -+ Other types are non-tagged type. - -To determine whether a `PyVar` is of a specific type, -you can use the following functions: - -+ `bool is_type(PyVar obj, Type type)` -+ `bool is_int(PyVar obj)` -+ `bool is_float(PyVar obj)` -+ `bool is_tagged(PyVar obj)` - -```cpp -PyVar obj = py_var(vm, 1); - -bool ok = is_type(obj, vm->tp_int); // true -ok = is_int(obj); // true -ok = is_tagged(obj); // true - -ok = is_type(obj, vm->tp_float); // false -ok = is_float(obj); // false -``` - -Simply put, `is_type` is the most general function and can check any types. -Other variants are designed for specific types and are faster. - -You can also use `check_` prefix functions assert the type of a `PyVar`, -which will throw `TypeError` on failure. - -+ `void check_type(PyVar obj, Type type)` - diff --git a/docs/quick-start/misc.md b/docs/quick-start/misc.md deleted file mode 100644 index e5925e8a..00000000 --- a/docs/quick-start/misc.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -icon: dot -label: 'Miscellaneous' -order: 0 ---- - -## The scope lock of gc - -Sometimes you need to use the following code to prevent the gc from collecting objects. - -```cpp -auto _lock = vm->gc_scope_lock(); -``` - -The scope lock is required if you create a PyVar and then try to run python-level bytecodes. - -For example, you create a temporary object on the stack and then call `vm->py_next`. - -```cpp -void some_func(VM* vm){ - PyVar obj = VAR(List(5)); - // unsafe - PyVar iter = vm->py_iter(obj); - PyVar next = vm->py_next(iter); -} -``` - -Because users can have an overload of `__next__`, this call is unsafe. -When the vm is running python-level bytecodes, gc may start and delete your temporary object. - -The scope lock prevents this from happening. - -```cpp -void some_func(VM* vm){ - PyVar obj = VAR(List(5)); - // safe - auto _lock = vm->gc_scope_lock(); - PyVar iter = vm->py_iter(obj); - PyVar next = vm->py_next(iter); -} -``` diff --git a/docs/quick-start/modules.md b/docs/quick-start/modules.md deleted file mode 100644 index 241c0a47..00000000 --- a/docs/quick-start/modules.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -icon: dot -label: 'Create Modules' -order: 50 ---- - -Modules are stored in `vm->_modules` and `vm->_lazy_modules`. -They are both dict-like objects. - -### Lazy modules - -A lazy module is a python source file. -It is compiled and executed when it is imported. -Use `[]` operator to add a lazy module. - -```cpp -vm->_lazy_modules["test"] = "pi = 3.14"; -``` - -```python -import test -print(test.pi) # 3.14 -``` - -### Native modules - -A native module is a module written in c++ or mixed c++/python. -Native modules are always compiled and executed when the VM is created. - -To creata a native module, use `vm->new_module(Str name)`. - -```cpp -PyObject* mod = vm->new_module("test"); -mod->attr().set("pi", py_var(vm, 3.14)); - -vm->bind(mod, "add(a: int, b: int)", - [](VM* vm, ArgsView args){ - int a = py_cast(vm, args[0]); - int b = py_cast(vm, args[1]); - return py_var(vm, a + b); - }); -``` - -```python -import test -print(test.pi) # 3.14 -print(test.add(1, 2)) # 3 -``` - -### Module resolution order - -When you do `import` a module, the VM will try to find it in the following order: - -1. Search `vm->_modules`, if found, return it. -2. Search `vm->_lazy_modules`, if found, compile and execute it, then return it. -3. Try `vm->_import_handler`. - - -### Customized import handler - -You can use `vm->_import_handler` to provide a custom import handler for the 3rd step. - -### Import module via cpp - -You can use `vm->py_import` to import a module. -This is equivalent to `import` in python. -Return the module object if success.