Make Your Own Python Pip Package

Published in

KoçSistem

9 min readNov 5, 2021

How can we library a programmed python code? You can find the answer to this question in this article. In this study, the methods of how we can do it will be explained step by step.

You are able to import your library and run your code in compatible environments without copying the source code. Other libraries may even be developed using the installed PyPi package.

· PyPi — The Python Package Index

· Get Your Code Ready For Publishing

· Get Pip Environment Prepare

· Create PyPi Config Files At The Project

· Create Account on PyPi

· Generate Distribution File

· Install To Local And Upload PyPi

· Uploading To Python Package Index

PyPi — The Python Package Index

Thousands of great, ready to use libraries with simple command instructions are available with PyPi. If you’re already using any of the PyPi packages, you probably have thought about how to convert your own code into a library.

After the upload process, you can easily share the solution you wrote through the PyPi index. Now, You can follow how we upload our own code to PyPi.

Get Your Code Ready For Publishing

Make a python project to the library.

Use code repository(GitHub, Bitbucket etc..) for source code
Clean unreferenced code in project
Remove all “print” statements from your code. You can use Logging package instead of print. Check out my previous article for more information on logging.
Check the name of your package. For this one, your package name able to be validated with pip-name PyPi package.

Install

$ pip install pip-name

Usage

$ pip-name <your_package_name>

Note: name is the distribution name of your package. This can be any name as long as it only contains letters, numbers, _ , and -. It also must not already be taken on pypi.org.

Be sure to update this with your username, as this ensures you won’t try to upload a package with the same name as one which already exists.

Transport your codes to {package_name} folder in src/{package_name} folder structure.
The package name you prefer should be the same as the library code to be imported.(this is not must)
Create a file called __init__.py under folder in src/{package_name}

_init_.py

# able to use for package version__version__ = "0.2.0"from package_name.Filename import Class_name,Class_name3from package_name.Filename2 import Class_name2# orfrom package_name.Filename3 import *

Get Pip Environment Prepare

Pip is a Python package manager. Check here for installation. if you already installed on your OS System.

if your OS System is MacOS. you can prefer brew package manager for pip installer.

MacOS Pip install with Brew

brew install python

If you haven’t installed anything python on your Mac OS system, the default Python environment will be 2.7. The python3 environment will be installed when you install pip install via Brew. If you do not change the python environment path, when you run the python code, it will run with version 2.7.

So you should start with the python3 command to use your pip install package installations.

Python Default Environment on Mac

After the install Python with Brew

Do not neglect the following procedure.

PyPi command on Terminal

python3 -m pip install --upgrade pip

Finally, make sure you have installed setuptools, wheel packages

install setuptools

pip install setuptools

install wheel

pip install wheel

Create PyPi Config Files At The Project

Add files that are used to prepare the project for distribution. All of these files to be created must be located in the root’ directory of the project.

PyPi needs some files to run:

pyproject.toml
setup.py / setup.cfg
LICENSE
README.md
MANIFEST.in

Now, let’s go through these files one by one.

Create File : pyproject.toml

In the pyproject.toml file, the required build tools (like pip and build) are defined what is needed to build your project. Those in this “requires” list will only use what is needed at build time, not after your package is installed.

pyproject.toml

[build-system]requires = ["setuptools>=46","wheel"]build-backend = "setuptools.build_meta"

My Python build system is “setuptools.build_meta”. If you were to use a different build system, you should define it whit build-backend.

Create File : setup

setup.cfg

[metadata]version = attr: package_name.__version__license_files = LICENSE

There are two types of metadata: static and dynamic.

· Static metadata (setup.cfg): guaranteed to be the same every time. This is simpler, easier to read, and avoids many common errors, like encoding errors.

· Dynamic metadata (setup.py): possibly non-deterministic. Any items that are dynamic or determined at install-time, as well as extension modules or extensions to setuptools, need to go into setup.py.

The following version information can be an example of dynamic metadata.

The package_name.__version__ value can be obtained from the version variable defined in the __init__.py file under the package_name directory. This process performs the dynamically sending values to setup.py file from outside.

It is recommended to use static metadata (setup.cfg).

My example is implemented with dynamic method

setup.py

import setuptoolswith open('README.md', 'r', encoding='utf-8') as fh:long_description = fh.read()setuptools.setup(name='codegenlib',author='Umut Boz',author_email='umut.boz@outlook.com',version="0.2.0",description='Code Generation library written by python. can use bash script, can be extend python code, can use mustache files or can use any string content for any code generation structure.',keywords='code generation, file generation, pypi, package',long_description=long_description,long_description_content_type='text/markdown',url='https://github.com/umutboz/code-gen-lib',project_urls={'Documentation': 'https://github.com/umutboz/code-gen-lib','Bug Reports':'https://github.com/umutboz/code-gen-lib/issues','Source Code': 'https://github.com/umutboz/code-gen-lib'},package_dir={'': 'src'},packages=setuptools.find_packages(where='src'),classifiers=[# see https://pypi.org/classifiers/'Development Status :: 5 - Production/Stable','Intended Audience :: Developers','Topic :: Software Development :: Build Tools','Programming Language :: Python :: 2.7','Programming Language :: Python :: 3','Programming Language :: Python :: 3.6','Programming Language :: Python :: 3.7','Programming Language :: Python :: 3.8','Programming Language :: Python :: 3.9','License :: OSI Approved :: MIT License','Operating System :: OS Independent',],python_requires='>=2.7',# install_requires=['Pillow'],extras_require={'dev': ['check-manifest'],# 'test': ['coverage'],},# entry_points={#     'console_scripts': [  # This can provide executable scripts#         'run=examplepy:main',# You can execute `run` in bash to run `main()` in src/examplepy/__init__.py#     ],# },)

The library I have developed supports python version 2.7. So I defined on field of python_requires.

For additional information on versioning, I recommend you review the document here.

Install_requires

Here, you define all the dependencies your package has — all the pip packages that you are importing. If there is a pip package(s) that you import in the code you develop, you should add them to this list.

Create File : README.md

You should explain your package’s usage documents and scenarios, use cases to developers who will use your package. Enter the following content in README.md. We added at setup.py. If it can’t find it, it will throw an error.

On the other hand, this readme content will appear as PyPi package information.

Create File : LICENSE

This is an example of an MIT license. It’s important for every package uploaded to the Python Package Index to include a license. Again this file is referenced in setup.py. This tells users who install your package the terms under which they can use your package.

LICENSE

Copyright (c) 2018 The Python Packaging AuthorityPermission is hereby granted, free of charge, to any person obtaining a copyof this software and associated documentation files (the "Software"), to dealin the Software without restriction, including without limitation the rightsto use, copy, modify, merge, publish, distribute, sublicense, and/or sellcopies of the Software, and to permit persons to whom the Software isfurnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in allcopies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THEAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHERLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THESOFTWARE.

Create File : MANIFEST.in

A MANIFEST.in file consists of commands, one per line, instructing setuptools to add or remove some set of files from the sdist.

Adding & removing files to & from the source distribution is done by writing a MANIFEST.in file at the project root. This is how we add unit tests

setup.cfg

include pyproject.tomlinclude *.mdinclude LICENSErecursive-include tests test*.py

Create Folder: Tests

Create a folder called tests in the root of your project.

pylint pytest

pip install pylint pytest

test_sample.py

import unittest
from src.your_package.lib_file import Class1class TestSimple(unittest.TestCase):def test_module_path(self):path = Class1.getModulePath()print(path)self.assertEqual(testModule.name, "test-module")
if __name__ == '__main__':unittest.main()

Create Generate Files And Folders

we will have some generated files and folders. I will address these later. We will generate them with some commands

Finally, Project Structure

The project structure will look like this:

root/├── LICENSE├── pyproject.toml├── README.md├── setup.cfg|setup.py├── src/│   └── package_name/│       ├── __init__.py│       └── lib_file1.py│       └── lib_file2.py└── tests/

Create Account on PyPi

You can register yourself for a PyPi account here.

Go to Account settings § API tokens, “Add API token”. The PyPI token only appears once, copy it somewhere.

If you are going to send PyPi packages via GitHub action, add your project’s settings>secrets>”New repository secret” naming of token “PYPI_API_TOKEN. But we don’t use it. Token is still needed but locally they will do publishing.

Push or release

The example package has automated tests and upload (publishing) already set up with GitHub Actions:

Every time you git push your master or main branch, the package is automatically tested against the desired Python versions with GitHub Actions.
Every time a new release (either the initial version or an updated version) is created, the package is automatically uploaded to PyPI with GitHub Actions.

Generate Distribution File

Make sure it’s up to date. Install or upgrade setuptools and wheel:

update setuptools and wheel

python3 -m pip install --user --upgrade setuptools wheel

Run the following command, in order to generate production version for source distribution (sdist) in dist folder:

Note running this command in project root directory

update setuptools and wheel

python3 setup.py sdist bdist_wheel

A dist folder will be created in the root directory. The tar.gz file is a source archive whereas the .whl file is a built distribution.

generated dist folder files

dist/{your_package_name}-{version}-py3-none-any.whl{your_package_name}-{version}.tar.gz

Install To Local And Upload PyPi

Run the following command, Install to local

install to Local PyPi

pip install dist/{package_name}-{version}.tar.gz

(You may need to uninstall existing package first:

install to Local PyPi

pip uninstall your_package

There may be several installed packages with the same name, so run pip uninstall multiple times until it says no more package to remove.)

Uploading To Python Package Index

Finally, it’s time to upload your package to the PyPi!

Install Twine

Install or upgrade Twine:

install to Local PyPi

python3 -m pip install --user --upgrade twine

To securely upload your project, you’ll need a PyPI API token. we have created before .. We have mentioned in the above steps.

Create a .pypirc file in your $HOME (~) directory. Users/{user} under For Mac its content should be:

~./pypirc

[pypi]username = __token__password = your PyPI token with start prefix pypi-

if you don’t manually create $HOME/.pypirc, you will be prompted for a username (which should be __token__) and password (which should be your PyPI token) when you run Twine)

Upload Twine

Run Twine to upload all of the archives under dist folder:

install to Local PyPi

python3 -m twine upload --repository pypi dist/*

after upload twine outputs

Uploading distributions to https://upload.pypi.org/legacy/Uploading codegenlib-0.1.0-py3-none-any.whl100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11.7k/11.7k [00:02<00:00, 5.06kB/s]Uploading codegenlib-0.2.0-py3-none-any.whl100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 24.9k/24.9k [00:01<00:00, 14.9kB/s]Uploading codegenlib-0.1.0.tar.gz100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13.0k/13.0k [00:01<00:00, 7.38kB/s]Uploading codegenlib-0.2.0.tar.gz100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 22.1k/22.1k [00:01<00:00, 19.5kB/s]View at:https://pypi.org/project/codegenlib/0.2.0/https://pypi.org/project/codegenlib/0.1.0/

Congratulations, you’ve packaged and distributed a Python project!

Finally, you can look at PyPi https://pypi.org/search/?q=your_uploaded_package

https://pypi.org/project/your_uploaded_package/

I am sharing my library as an example, you can review it. https://github.com/umutboz/code-gen-lib