...

2025-10-20 11:30:18 +00:00 · 2024-08-11 23:52:41 +08:00 · 2024-08-11 23:52:41 +08:00 · f22dda3125
commit f22dda3125
parent 2de9efc0e9
14 changed files with 4 additions and 766 deletions
--- a/docs/C-API/introduction.md
+++ b/docs/C-API/introduction.md
@ -4,36 +4,4 @@ icon: dot
 order: 10
 ---
-### What C-API is for
+TBA
 If your target platform does not support C++17. You can compile pkpy into a static library and use its exported C-APIs.
 Our C-APIs take a lot of inspiration from the lua C-APIs.
 Methods return a `bool` indicating if the operation succeeded or not.
 Special thanks for [@koltenpearson](https://github.com/koltenpearson)'s contribution.
 !!!
 C-APIs are always stable and backward compatible.
 !!!
 ### Basic functions
 + `pkpy_vm* pkpy_new_vm(bool enable_os)`
    Wraps `new VM(enable_os)` in C++.
 + `void pkpy_delete_vm(pkpy_vm*)`
    Wraps `delete vm` in C++.
 + `bool pkpy_exec(pkpy_vm*, const char* source)`
    Wraps `vm->exec`. Execute a string of source code.
 + `bool pkpy_exec_2(pkpy_vm*, const char* source, const char* filename, int mode, const char* module)`
    Wraps `vm->exec_2`. Execute a string of source code with more options.
 + `void pkpy_set_main_argv(pkpy_vm*, int argc, char** argv)`
    Wraps `vm->set_main_argv`. Set the `sys.argv` before executing scripts.
--- a/docs/C-API/stack.md
+++ b/docs/C-API/stack.md
@ -1,169 +0,0 @@
 ---
 title: Stack Manipulation
 icon: dot
 order: 8
 ---
 ### Basic manipulation
 + `bool pkpy_dup(pkpy_vm*, int)`
    Duplicate the value at the given index.
 + `bool pkpy_pop(pkpy_vm*, int)`
    Pop `n` values from the stack.
 + `bool pkpy_pop_top(pkpy_vm*)`
    Pop the top value from the stack.
 + `bool pkpy_dup_top(pkpy_vm*)`
    Duplicate the top value on the stack.
 + `bool pkpy_rot_two(pkpy_vm*)`
    Swap the top two values on the stack.
 + `int pkpy_stack_size(pkpy_vm*)`
    Get the element count of the stack.
 ### Basic push, check and convert
 + `pkpy_push_xxx` pushes a value onto the stack.
 + `pkpy_is_xxx` checks if the value at the given index is of the given type.
 + `pkpy_to_xxx` converts the value at the given index to the given type.
 Stack index is 0-based instead of 1-based. And it can be negative, which means the index is counted from the top of the stack.
 ```c
 // int
 PK_EXPORT bool pkpy_push_int(pkpy_vm*, int);
 PK_EXPORT bool pkpy_is_int(pkpy_vm*, int i);
 PK_EXPORT bool pkpy_to_int(pkpy_vm*, int i, int* out);
 // float
 PK_EXPORT bool pkpy_push_float(pkpy_vm*, double);
 PK_EXPORT bool pkpy_is_float(pkpy_vm*, int i);
 PK_EXPORT bool pkpy_to_float(pkpy_vm*, int i, double* out);
 // bool
 PK_EXPORT bool pkpy_push_bool(pkpy_vm*, bool);
 PK_EXPORT bool pkpy_is_bool(pkpy_vm*, int i);
 PK_EXPORT bool pkpy_to_bool(pkpy_vm*, int i, bool* out);
 // string
 PK_EXPORT bool pkpy_push_string(pkpy_vm*, pkpy_CString);
 PK_EXPORT bool pkpy_is_string(pkpy_vm*, int i);
 PK_EXPORT bool pkpy_to_string(pkpy_vm*, int i, pkpy_CString* out);
 // void_p
 PK_EXPORT bool pkpy_push_voidp(pkpy_vm*, void*);
 PK_EXPORT bool pkpy_is_voidp(pkpy_vm*, int i);
 PK_EXPORT bool pkpy_to_voidp(pkpy_vm*, int i, void** out);
 // none
 PK_EXPORT bool pkpy_push_none(pkpy_vm*);
 PK_EXPORT bool pkpy_is_none(pkpy_vm*, int i);
 ```
 ### Special push
 + `pkpy_push_null(pkpy_vm*)`
    Push a `PY_NULL` onto the stack. It is used for `pkpy_vectorcall`.
 + `pkpy_push_function(pkpy_vm*, const char* sig, pkpy_CFunction f)`
    Push a function onto the stack. `sig` is the function signature, e.g. `add(a: int, b: int) -> int`. `f` is the function pointer.
 + `pkpy_push_module(pkpy_vm*, const char* name)`
    Push a new module onto the stack. `name` is the module name. This is not `import`. It creates a new module object.
 ### Variable access
 + `bool pkpy_getattr(pkpy_vm*, pkpy_CName name)`
    Push `b.<name>` onto the stack. Return false if the attribute is not found.
    ```
    [b] -> [b.<name>]
    ```
 + `bool pkpy_setattr(pkpy_vm*, pkpy_CName name)`
    Set `b.<name>` to the value at the top of the stack.
    First push the value, then push `b`.
    ```
    [value, b] -> []
    ```
 + `bool pkpy_getglobal(pkpy_vm*, pkpy_CName name)`
    Push a global/builtin variable onto the stack. Return false if the variable is not found.
    ```
    [] -> [value]
    ```
 + `bool pkpy_setglobal(pkpy_vm*, pkpy_CName name)`
    Set a global variable to the value at the top of the stack.
    ```
    [value] -> []
    ```
 + `bool pkpy_eval(pkpy_vm*, const char* source)`
    Evaluate a string and push the result onto the stack.
    ```
    [] -> [result]
    ```
 + `bool pkpy_unpack_sequence(pkpy_vm*, int size)`
    Unpack a sequence at the top of the stack. `size` is the element count of the sequence.
    ```
    [a] -> [a[0], a[1], ..., a[size - 1]]
    ```
 + `bool pkpy_get_unbound_method(pkpy_vm*, pkpy_CName name)`
    It is used for method call.
    Get an unbound method from the object at the top of the stack. `name` is the method name.
    Also push the object as self.
    ```
    [obj] -> [obj.<name> self]
    ```
 + `bool pkpy_py_repr(pkpy_vm*)`
    Get the repr of the value at the top of the stack.
    ```
    [value] -> [repr(value)]
    ```
 + `bool pkpy_py_str(pkpy_vm*)`
    Get the str of the value at the top of the stack.
    ```
    [value] -> [str(value)]
    ```
 + `bool pkpy_py_import(pkpy_vm*, const char* name)`
    Import a module and push it onto the stack.
    ```
    [] -> [module]
    ```
--- a/docs/modules/c.md
+++ b/docs/modules/c.md
@ -1,8 +0,0 @@
 ---
 icon: package
 label: c
 ---
 Interop with pointers and C structs.
 https://github.com/pocketpy/pocketpy/blob/main/include/typings/c.pyi
--- a/docs/modules/colorsys.md
+++ b/docs/modules/colorsys.md
@ -1,8 +0,0 @@
 ---
 icon: package
 label: colorsys
 ---
 The same as the standard library module `colorsys` in python 3.11.
 https://github.com/python/cpython/blob/3.11/Lib/colorsys.py
--- a/docs/modules/csv.md
+++ b/docs/modules/csv.md
@ -1,33 +0,0 @@
 ---
 icon: package
 label: csv
 ---
 ### `csv.reader(csvfile: list[str]) -> list[list]`
 Parse a CSV file into a list of lists.
 ### `csv.DictReader(csvfile: list[str]) -> list[dict]`
 Parse a CSV file into a list of dictionaries.
 ## Example
 ```python
 import csv
 data = """a,b,c
 1,2,3
 """
 print(csv.reader(data.splitlines()))
 # [
 #    ['a', 'b', 'c'],
 #    ['1', '2', '3']
 # ]
 print(csv.DictReader(data.splitlines()))
 # [
 #    {'a': '1', 'b': '2', 'c': '3'}
 # ]
 ```
--- a/docs/quick-start/installation.md
+++ b/docs/quick-start/installation.md
@ -1,7 +1,7 @@
 ---
-icon: dot
+icon: rocket
-label: 'Installation'
+order: 20
-order: 100
+label: Quick Start
 ---
 You have two options to integrate pkpy into your project.
@ -126,55 +126,3 @@ __ERROR:
    return 1;
 }
 ```
 ### Overview
 pkpy's C++ interfaces are organized in an object-oriented way.
 All classes are located in `pkpy` namespace.
 The most important class is the `VM` class. A `VM` instance is a python virtual machine which holds all necessary runtime states, including callstack, modules, variables, etc.
 A process can have multiple `VM` instances. Each `VM` instance is independent from each other.
 !!!
 Always use C++ `new` operator to create a `VM` instance.
 DO NOT declare it on the stack. It may cause stack overflow.
 !!!
 ```cpp
 VM* vm = new VM();
 ```
 The constructor can take 1 extra parameters.
 #### `VM(bool enable_os=true)`
 + `enable_os`, whether to enable OS-related features or not. This setting controls the availability of privileged modules such os `io` and `os` as well as builtin function `open`. **It is designed for sandboxing.**
 When you are done with the `VM` instance, use `delete` operator to dispose it.
 ```cpp
 delete vm;
 ```
 ### Hook standard buffer
 By default, pkpy outputs all messages and errors to `stdout` and `stderr`.
 You can redirect them to your own buffer by setting `vm->_stdout` and `vm->_stderr`.
 These two fields are C function pointers with the following signature:
 ```cpp
 void(*)(const char*, int);
 ```
 Or you can override these two virtual functions:
 ```cpp
    virtual void stdout_write(const Str& s){
        _stdout(s.data, s.size);
    }
    virtual void stderr_write(const Str& s){
        _stderr(s.data, s.size);
    }
 ```
--- a/docs/quick-start/attr.md
+++ b/docs/quick-start/attr.md
@ -1,72 +0,0 @@
 ---
 icon: dot
 label: 'Access Attributes'
 order: 80
 ---
 ### Direct access
 Some python objects have an instance dict, a.k.a, `__dict__` in cpython.
 You can use `obj->attr()` to manipulate the instance dict of an object.
 ```cpp
 VM* vm = new VM();
 // get the `builtin` module
 PyVar builtins = vm->builtins;
 // get `dict` type
 PyVar dict = builtins->attr("dict");
 // set `pi = 3.14`
 builtins->attr().set("pi", py_var(vm, 3.14));
 ```
 However, you cannot call `attr` on an object which does not have an instance dict.
 For example, the `int` object.
 ```cpp
 // create a `int` object
 PyVar obj = py_var(vm, 1);
 // THIS IS WRONG!! WILL LEAD TO A SEGFAULT!!
 PyVar add = obj->attr("__add__");
 ```
 To determine whether an object has instance dict or not, you can use this snippet.
 ```cpp
 // 1. call `is_tagged` to check the object supports `->` operator
 // 2. call `is_attr_valid` to check the existence of instance dict
 PyVar obj = py_var(vm, 1);
 bool ok = !is_tagged(obj) && obj->is_attr_valid();  // false
 ```
 ### General access
 As you can see, direct access does not take care of derived attributes or methods.
 In most cases, what you need is `getattr` and `setattr`.
 These two methods handle all possible cases.
 #### `PyVar getattr(PyVar obj, StrName name, bool throw_err=true)`
 This method is equivalent to `getattr` in python.
 If the attribute is not found, it will return `nullptr`
 or throw an `AttributeError` depending on the value of `throw_err`.
 ```cpp
 // create a `int` object
 PyVar obj = py_var(vm, 1);
 // get its `__add__` method, which is a `bound_method` object
 PyVar add = vm->getattr(obj, "__add__");
 // call it (equivalent to `1 + 2`)
 PyVar ret = vm->call(add, py_var(vm, 2););
 // get the result
 int result = py_cast<int>(vm, ret);
 std::cout << result << std::endl; // 3
 ```
 #### `void setattr(PyVar, StrName, PyVar)`
 This method is equivalent to `setattr` in python.
 It raises `TypeError` if the object does not support attribute assignment.
--- a/docs/quick-start/call.md
+++ b/docs/quick-start/call.md
@ -1,71 +0,0 @@
 ---
 icon: dot
 label: 'Call Python Function'
 order: 70
 ---
 pkpy uses a variant of the [Vectorcall](https://peps.python.org/pep-0590/) protocol (PEP 590).
 You can use `call` to invoke any python callable object,
 including functions, methods, classes, etc.
 For methods, `call_method` can be used.
 + `PyVar call(PyVar obj, ...)`
 + `PyVar call_method(PyVar obj, StrName name, ...)`
 ### Example
 Let's create a `dict` object and set a key-value pair,
 which equals to the following python snippet.
 ```python
 obj = {}        # declare a `dict`
 obj["a"] = 5    # set a key-value pair
 print(obj["a"]) # print the value
 ```
 First, create an empty dict object,
 ```cpp
 PyVar tp = vm->builtins->attr("dict");
 PyVar obj = vm->call(tp);	// this is a `dict`
 ```
 And set a key-value pair,
 ```cpp
 PyVar _0 = py_var(vm, "a");
 PyVar _1 = py_var(vm, 5);
 vm->call_method(obj, "__setitem__", _0, _1);
 ```
 And get the value,
 ```cpp
 PyVar ret = vm->call_method(obj, "__getitem__", _0);
 std::cout << py_cast<int>(vm, i64);
 ```
 If you want to call with dynamic number of arguments,
 you should use `vm->vectorcall`. This is a low-level, stack-based API.
 1. First push the callable object to the stack.
 2. Push the `self` object to the stack. If there is no `self`, push `PY_NULL`.
 3. Push the arguments to the stack.
 4. Call `vm->vectorcall` with the number of arguments.
 ```cpp
 PyVar f_sum = vm->builtins->attr("sum");
 List args(N);   // a list of N arguments
 vm->s_data.push_back(f_sum);
 vm->s_data.push_back(PY_NULL);  // self
 for(PyVar arg : args) {
    vm->s_data.push_back(arg);
 }
 PyVar ret = vm->vectorcall(args.size());
 ```
--- a/docs/quick-start/config.md
+++ b/docs/quick-start/config.md
@ -1,30 +0,0 @@
 ---
 icon: dot
 label: 'Advanced Config'
 order: -2
 ---
 ### Enable os-related features
 If you want to enable os-related features, you can do this before including `pocketpy.h`.
 ```cpp
 #define PK_ENABLE_OS 1
 #include <pocketpy.h>
 ```
 ### Working with multiple threads
 pkpy does not support multi-threading. But you can create multiple `VM` instances and run them in different threads.
 You can do the following to ensure thread safety for `VM` instances:
 ```cpp
 #define PK_ENABLE_THREAD 1
 #include <pocketpy.h>
 ```
 ### Full config
 You can take a look at `include/pocketpy/config.h` to see all the available configuration macros.
--- a/docs/quick-start/exec.md
+++ b/docs/quick-start/exec.md
@ -1,59 +0,0 @@
 ---
 icon: dot
 label: 'Execute Python Code'
 order: 93
 ---
 ### Simple execution
 Once you have a `VM` instance, you can execute python code by calling `exec` method.
 #### `PyVar exec(Str source, Str filename, CompileMode mode, PyVar _module=nullptr)`
 + `source`, the python source code to be executed
 + `filename`, the filename of the source code. This is used for error reporting
 + `mode`, the compile mode. See below for details
 + `module`, the module where the code will be executed. If `nullptr`, the code will be executed in the `__main__` module
 `exec` handles possible exceptions and returns a `PyVar`.
 If the execution is not successful, e.g. a syntax error or a runtime exception,
 the return value will be `nullptr`.
 There are also overloaded versions of `exec` and `eval`, which is useful for simple execution:
 + `PyVar exec(Str source)`
 + `PyVar eval(Str source)`
 ### Compile mode
 The `mode` parameter controls how the source code is compiled. There are 5 possible values:
 + `EXEC_MODE`, this is the default mode. Just do normal execution.
 + `EVAL_MODE`, this mode is used for evaluating a single expression. The `source` should be a single expression. It cannot contain any statements.
 + `REPL_MODE`, this mode is used for REPL. It is similar to `EXEC_MODE`, but generates `PRINT_EXPR` opcode when necessary.
 + `CELL_MODE`, this mode is designed for Jupyter like execution. It is similar to `EXEC_MODE`, but generates `PRINT_EXPR` opcode when necessary.
 + `JSON_MODE`, this mode is used for JSON parsing. It is similar to `EVAL_MODE`, but uses a lexing rule designed for JSON.
 ### Fine-grained execution
 In some cases, you may want to execute python code in a more fine-grained way.
 These two methods are provided for this purpose:
 + `CodeObject_ compile(Str source, Str filename, CompileMode mode, bool unknown_global_scope)`
 + `PyVar _exec(CodeObject_ co, PyVar _module)`
 1. `compile` compiles the source code into a `CodeObject_` instance. Leave `unknown_global_scope` to `false` if you don't know what it means.
 2. `_exec` executes the `CodeObject_` instance.
 !!!
 `_exec` does not handle exceptions, you need to use `try..catch` manually.
 !!!
 ```cpp
 try{
    CodeObject_ code = vm->compile("a[0]", "main.py", EXEC_MODE, false);
    vm->_exec(code, vm->_main);
 }catch(TopLevelException e){
    // use e.summary() to get a summary of the exception
    std::cerr << e.summary() << std::endl;
 }
 ```
--- a/docs/quick-start/index.yml
+++ b/docs/quick-start/index.yml
@ -1,3 +0,0 @@
 icon: rocket
 order: 20
 label: Quick Start
--- a/docs/quick-start/interop.md
+++ b/docs/quick-start/interop.md
@ -1,117 +0,0 @@
 ---
 icon: dot
 label: 'Interop with PyVar'
 order: 90
 ---
 In pkpy, any python object is represented by a `PyVar`.
 ### Create `PyVar` from C type
 A set of overloaded function `PyVar py_var(VM* vm, ...)` were implemented to
 create a `PyVar` from a supported C type.
 Assume we have a `VM* vm` instance.
 You can create a python `int` object from a C `i64` type:
 ```cpp
 i64 i = 2;
 PyVar obj = py_var(vm, i);
 ```
 Each python type has a corresponding C type, for example, `int` in python is `i64` in C.
 python's `list` corresponds to `List`, `str` corresponds to `Str`, etc.
 For strings, we have defined
 a set of overloaded version including `const char*`, `std::string`, `std::string_view`, `Str`, etc.
 ```cpp
 PyVar obj = py_var(vm, "abc");		// create a python str object
 ```
 A more complex example is to create a python `list`.
 In the following code, we create a `list` equals to `[0, 1, 2, 3]`.
 ```cpp
 List list;
 for (i64 i = 0; i < 4; i++) {
    list.push_back(py_var(vm, i));
 }
 obj = py_var(vm, std::move(list));		// create a python list object
 ```
 Please note that `std::move` is used here to avoid unnecessary copy.
 Most types have both a rvalue and a lvalue version of `py_var` function.
 ### Access internal C type of `PyVar`
 A set of template function `T py_cast<T>(VM* vm, PyVar obj)` were implemented.
 ```cpp
 i64 i = 2;
 PyVar obj = py_var(vm, i);
 // cast a PyVar to C i64
 i64 j = py_cast<i64>(vm, obj);
 ```
 The `py_cast` function will check the type of `obj` before casting.
 If the type is not matched, a `TypeError` will be thrown.
 However, this type check has a cost. If you are sure about the type of `obj`,
 you can use the underscore version `_py_cast` to skip the type check.
 ```cpp
 // cast a PyVar to C i64 (unsafe but faster)
 i64 j = _py_cast<i64>(vm, obj);		
 ```
 For complex objects like `list`, we can use reference cast to avoid unnecessary copy.
 ```cpp
 PyVar obj = py_var(vm, List());
 // reference cast (no copy)
 List& list = py_cast<List&>(vm, obj);
 ```
 ### Check type of `PyVar`
 Each `PyVar` has a `Type type` field to indicate its type.
 `Type` is just an integer which is the global index in `vm->_all_types`.
 `VM` class has a set of predefined `Type` constants for quick access.
 They are prefixed by `tp_`. For example, `tp_object`(object),
 `tp_int`(int), `tp_str`(str), `tp_list`(list), etc.
 Types are divided into **tagged type** and **non-tagged type**.
 + small `int` objects are tagged.
 + Other types are non-tagged type.
 To determine whether a `PyVar` is of a specific type,
 you can use the following functions:
 + `bool is_type(PyVar obj, Type type)`
 + `bool is_int(PyVar obj)`
 + `bool is_float(PyVar obj)`
 + `bool is_tagged(PyVar obj)`
 ```cpp
 PyVar obj = py_var(vm, 1);
 bool ok = is_type(obj, vm->tp_int);		// true
 ok = is_int(obj);						// true
 ok = is_tagged(obj);					// true
 ok = is_type(obj, vm->tp_float);		// false
 ok = is_float(obj);						// false
 ```
 Simply put, `is_type` is the most general function and can check any types.
 Other variants are designed for specific types and are faster.
 You can also use `check_` prefix functions assert the type of a `PyVar`,
 which will throw `TypeError` on failure.
 + `void check_type(PyVar obj, Type type)`
--- a/docs/quick-start/misc.md
+++ b/docs/quick-start/misc.md
@ -1,41 +0,0 @@
 ---
 icon: dot
 label: 'Miscellaneous'
 order: 0
 ---
 ## The scope lock of gc
 Sometimes you need to use the following code to prevent the gc from collecting objects.
 ```cpp
 auto _lock = vm->gc_scope_lock();
 ```
 The scope lock is required if you create a PyVar and then try to run python-level bytecodes.
 For example, you create a temporary object on the stack and then call `vm->py_next`.
 ```cpp
 void some_func(VM* vm){
    PyVar obj = VAR(List(5));
    // unsafe
    PyVar iter = vm->py_iter(obj);
    PyVar next = vm->py_next(iter);
 }
 ```
 Because users can have an overload of `__next__`, this call is unsafe.
 When the vm is running python-level bytecodes, gc may start and delete your temporary object.
 The scope lock prevents this from happening.
 ```cpp
 void some_func(VM* vm){
    PyVar obj = VAR(List(5));
    // safe
    auto _lock = vm->gc_scope_lock();
    PyVar iter = vm->py_iter(obj);
    PyVar next = vm->py_next(iter);
 }
 ```
--- a/docs/quick-start/modules.md
+++ b/docs/quick-start/modules.md
@ -1,67 +0,0 @@
 ---
 icon: dot
 label: 'Create Modules'
 order: 50
 ---
 Modules are stored in `vm->_modules` and `vm->_lazy_modules`.
 They are both dict-like objects.
 ### Lazy modules
 A lazy module is a python source file.
 It is compiled and executed when it is imported.
 Use `[]` operator to add a lazy module.
 ```cpp
 vm->_lazy_modules["test"] = "pi = 3.14";
 ```
 ```python
 import test
 print(test.pi)  # 3.14
 ```
 ### Native modules
 A native module is a module written in c++ or mixed c++/python.
 Native modules are always compiled and executed when the VM is created.
 To creata a native module, use `vm->new_module(Str name)`.
 ```cpp
 PyObject* mod = vm->new_module("test");
 mod->attr().set("pi", py_var(vm, 3.14));
 vm->bind(mod, "add(a: int, b: int)",
    [](VM* vm, ArgsView args){
        int a = py_cast<int>(vm, args[0]);
        int b = py_cast<int>(vm, args[1]);
        return py_var(vm, a + b);
    });
 ```
 ```python
 import test
 print(test.pi)  # 3.14
 print(test.add(1, 2))  # 3
 ```
 ### Module resolution order
 When you do `import` a module, the VM will try to find it in the following order:
 1. Search `vm->_modules`, if found, return it.
 2. Search `vm->_lazy_modules`, if found, compile and execute it, then return it.
 3. Try `vm->_import_handler`.
 ### Customized import handler
 You can use `vm->_import_handler` to provide a custom import handler for the 3rd step.
 ### Import module via cpp
 You can use `vm->py_import` to import a module.
 This is equivalent to `import` in python.
 Return the module object if success.