Translated by ChatGPT.

    Here, "others" also includes the version of yourself six months later, who may no longer remember how this code was written.

    Some time ago, I wrote an article called Using Import to Reference Existing Code, which explained how to use code written by others. I briefly mentioned a module's setup.py file at the end but didn’t go further. Now, let’s continue.

    Basic Process

    • Write your code and structure the project files in a specific way (see next section).
    • According to your project structure, fill in a pyproject.toml, setup.cfg, or setup.py file.
    • Install the build library, then run python -m build to create a dist/ folder with your built files. Optionally, upload the project to PyPi or Conda.

    Project File Structure

    Common file structures include the src-layout and flat-layout. Some small projects consist of a single Python file.

    src-layout

    In the src-layout, the folder containing the package source code has another folder above it, typically named src (although it could be named differently). The pyproject.toml file is on the same level as the src/ folder.

    <project_name> 
    ├── LICENSE 
    ├── pyproject.toml 
    ├── README.md 
    ├── src/ 
    │   └── <package_name>/
    │   ├── init.py 
    │   └── example.py 
    └── tests/

    flat-layout

    In a flat-layout, the folder containing the package source code is directly under the top level of the project, on the same level as pyproject.toml.

    This structure is older and not generally recommended.

    <project_name> 
    ├── pyproject.toml # and/or setup.cfg/setup.py
    ├── <package_name> 
    |   ├── init.py 
    |   └── ... (other Python files) 
    ├── test 
    |   └── ... (test files) 
    ├── README.md 
    └── LICENSE

    Single-File Project

    This can be considered a special case of flat-layout.

    <project_name> 
    ├── pyproject.toml # and/or setup.cfg/setup.py 
    ├── <my_module>.py 
    ├── README.rst or README.md 
    └── LICENSE

    Filling in pyproject.toml, setup.cfg, or setup.py File

    To package your code into an installable package, at least one of these three files must appear in the project root.

    The file must contain project information in the syntax specific to its extension, most of which is self-explanatory.

    Key parameters, such as name, packages, and package_dir, are based on the file structure. If the file structure fully follows the structure outlined in the previous section, then automatic discovery with setuptools.find_packages() should suffice.

    name

    This is a required parameter.

    Note: In the file structure from the previous section, there are two names: <project_name> and <package_name>.

    • <project_name> is the name of the development project as a whole. If you are using a version control system like git, this is the name of your repository.
    • <package_name> can—but doesn’t have to—be the name you import in other code. This import name is specified by the name parameter in pyproject.toml / setup.cfg / setup.py. It cannot contain hyphens and can only use underscores.

    If your name parameter differs from <package_name>, you must also set the package_dir parameter.

    There is also a third name: the one used for pip install __ when uploaded to PyPI. This name can contain hyphens, like scikit-image.

    packages

    This parameter is a list, but it typically takes the result of setuptools.find_packages().

    The find_packages() function commonly accepts three optional parameters:

    • where: A path relative to setup.py
    • include: A list of glob patterns
    • exclude: A list of glob patterns

    Leaving all parameters blank uses automatic discovery.

    package_dir

    This parameter is a dictionary with two main uses:

    • In a standard src-layout, simply write {"": "src/"}, indicating that all code is within this folder.
    • If the Python module structure differs from the code file structure, use this dictionary to define the mapping between modules and folders.

    File paths are relative to setup.py.

    py_modules

    This parameter is a list of file paths, mostly used for single-file structures.

    Building and Uploading

    Install the build tool: python3 -m pip install --upgrade build

    Run the build: python3 -m build

    This will create a dist/ folder containing the build files.

    To allow others to install your program using pip install, you need to upload the build to PyPI. Instructions are here.

    Installation

    Static Installation

    Packages already uploaded to PyPI can be installed with pip install <package>. This is called static installation.

    Dynamic Installation

    For packages still under development, run pip install -e . in the directory containing setup.py. This is called dynamic installation because changes in the code are reflected in the referencing projects in real time.

    Exercises

    Here is the file structure of a data analysis project, slightly modified:

    /path/to/project/directory/ 
    |-- notebooks/ 
        |-- 01-first-logical-notebook.ipynb 
        |-- 02-second-logical-notebook.ipynb 
        |-- prototype-notebook.ipynb 
        |-- archive/ 
            |-- no-longer-useful.ipynb 
    |-- src/ 
        |-- projectname/ 
            |-- init.py 
            |-- config.py 
            |-- data.py 
            |-- utils.py 
            |-- setup.py 
    |-- README.md 
    |-- data/ 
    |-- raw/ 
    |-- processed/ 
    |-- cleaned/ 
    |-- scripts/ 
        |-- script1.py 
        |-- script2.py 
        |-- archive/ 
            |-- no-longer-useful.py 
    |-- environment.yml

    Questions:

    1. Is this a flat-layout or src-layout?
    2. How should setup.py be written? What would be the output of pwd when executing a dynamic installation?

    See also: