gRPC API packages ================= Protobuf service definitions provide the API specification for underlying server implementations so that each consuming client library has a clear contract for gRPC data messages. Ideally, the Protobuf (``.proto``) files have a single repository established as the source of truth, organized by API version increment as the API definition expands and changes. Because most client libraries are custom implementations enhancing the developer experience when consuming the service, releasing the Protobuf definitions publicly gives full flexibility to developers to operate at the abstraction layer they choose. Maintain API definition repository ---------------------------------- Because the Protobuf definition of the service is language agnostic, the repository containing the PROTO files can be created within the top-level `Ansys GitHub organization`_. Every update of the PROTO files follows a standard pull request process as a sanity check for API definition accuracy. Language-specific packages can be generated for each merge or on a set cadence. Manage Protobuf definitions for Python clients ---------------------------------------------- Within Ansys, and more specifically in the PyAnsys environment, most client libraries have a dedicated Python package containing the needed PROTO files compiled as Python source code. These are typically consumed by the PyAnsys client libraries for communicating with their respective services. For example, `PyMAPDL`_ consumes the ``ansys-api-mapdl`` package, which is built in the `ansys-api-mapdl`_ repository. Build an ``ansys-api-`` repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Ansys GitHub organization has a dedicated template repository for creating PROTO file repositories and the needed files to generate the Python API packages to be consumed by the PyAnsys clients. To set up an API repository like the ``ansys-api-mapdl`` one, select the `ansys-api-template `_ repository when creating a repository within the Ansys GitHub organization. To understand how to use the ``ansys-api-template`` repository, see `Expected usage `_ in this repository's README. Build Python stub classes ~~~~~~~~~~~~~~~~~~~~~~~~~ The ``ansys-api-template`` repository uses the `ansys-tools-protoc-helper `_ utility to auto-generate Python wheels that can be consumed by downstream Python client libraries. To use the ``ansys-tools-protoc-helper`` utility, include it in the ``pyproject.toml`` file as a build dependency: .. code-block:: toml [build-system] requires = ["setuptools >= 42.0.0", "wheel", "ansys_tools_protoc_helper"] Then generate a Python wheel containing the autogenerated Python source with these commands: .. code-block:: bash pip install -U pip pip install build python -m build Publish the API package ~~~~~~~~~~~~~~~~~~~~~~~ `PyPI`_ is the common package manager where API packages are released. Here is an example of a workflow pipeline for building and publishing the Python stub package. In this example, the ``ansys-api-geometry`` workflow is shown. However, you can easily copy and adapt this workflow. Only the ``PYTHON_PACKAGE_IMPORT`` environment variable would have to be changed: .. code-block:: yaml name: GitHub CI on: pull_request: push: tags: - "*" branches: - main env: MAIN_PYTHON_VERSION: '3.10' PYTHON_PACKAGE_IMPORT: 'ansys.api.geometry.v0' jobs: build: name: Build package runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v4 with: python-version: ${{ env.MAIN_PYTHON_VERSION }} - name: Install build requirements run: | pip install -U pip pip install build - name: Build run: python -m build - name: Install run: pip install dist/*.whl - name: Test import run: | mkdir tmp cd tmp python -c "import ${{ env.PYTHON_PACKAGE_IMPORT }}; print('Successfully imported ${{ env.PYTHON_PACKAGE_IMPORT }}')" python -c "from import __version__; print(__version__)" - name: Upload packages uses: actions/upload-artifact@v3 with: name: ansys-api-package path: dist/ retention-days: 7 release: name: Release package if: github.event_name == 'push' && contains(github.ref, 'refs/tags') needs: [build] runs-on: ubuntu-latest steps: - name: Set up Python uses: actions/setup-python@v4 with: python-version: ${{ env.MAIN_PYTHON_VERSION }} - uses: actions/download-artifact@v3 - name: Display structure of downloaded files run: ls -R - name: Upload to Public PyPi run: | pip install twine twine upload --skip-existing ./**/*.whl twine upload --skip-existing ./**/*.tar.gz env: TWINE_USERNAME: __token__ TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} - name: Release uses: softprops/action-gh-release@v1 with: generate_release_notes: true files: | ./**/*.whl ./**/*.tar.gz ./**/*.pdf Version the API package ^^^^^^^^^^^^^^^^^^^^^^^ PyPI packages follow semantic versioning while gRPC Protobuf API versions typically follow a simplified ``v*`` versioning pattern. The PyPI package version is not expected to synchronize with the Protobuf API version, and multiple public APIs can be exposed simultaneously. For example, if you have a ``v0`` for MAPDL exposed, you can access it with this code: .. code:: python from ansys.api.mapdl.v0 import mapdl_pb2_grpc While if the API has a ``v1`` API exposed, a different library could also use: .. code:: python from ansys.api.mapdl.v1 import mapdl_pb2_grpc Ansys follows `Microsoft's gRPC versioning `_ recommendations, which stipulate that incrementing the gRPC Protobuf version is only necessary when making a backwards breaking change. Non-breaking changes include: * Adding a service * Adding a method to a service * Adding a field to a request message However, this only applies to the ``vN`` gRPC Protobuf API. Python packages tend to follow semantic versioning, and PyAnsys packages follow this approach. Therefore, these Python gRPC API packages should also follow semantic versioning. - Plan on releasing a new minor version when adding or removing features, messages, and services. - Plan on releasing a patch release when fixing bugs that do not change the behavior of the API. Only plan on releasing a major release once the API is stable and no major release is scheduled in the near future. This way, you can expose a ``v0`` and/or ``v1`` gRPC Protobuf API and release frequent updates using semantic versioning. Release the API package ^^^^^^^^^^^^^^^^^^^^^^^ As shown in the ``release`` section of the previous GitHub workflow, once the Python API package is compiled it is then uploaded to the public PyPI. In order to do so, it is necessary to have access to the ``PYPI_TOKEN`` within the GitHub repository. To get the needed credentials, contact the `PyAnsy core team `_. If the repository cannot be uploaded to the public PyPI yet but your Python client library needs to consume this Python API package, it can be uploaded to the private PyAnsys PyPI. For the required ``PYANSYS_PYPI_PRIVATE_PAT`` password, contact the `PyAnsy core team `_. In this last case, the ``Upload to Public PyPi`` workflow section should be replaced with the ``Upload to Private PyPi`` workflow section: .. code-block:: yaml - name: Upload to Private PyPi run: | pip install twine twine upload --skip-existing ./**/*.whl twine upload --skip-existing ./**/*.tar.gz env: TWINE_USERNAME: PAT TWINE_PASSWORD: ${{ secrets.PYANSYS_PYPI_PRIVATE_PAT }} TWINE_REPOSITORY_URL: https://pkgs.dev.azure.com/pyansys/_packaging/pyansys/pypi/upload Consume the API package within Python ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once the API package has been published to PyPI, you can include a reference within the client library build dependencies. For information on how to specify a project's required dependencies, see :ref:`Required Dependencies`. Use the API package within the Python client ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The stub imports follow a standard pattern. For each API service, there is a ``*_pb2`` module that defines all messages within a specific service file and a ``*_pb2_grpc`` module that defines a ``Stub`` class that encapsulates all service methods. Example gRPC imports within the wrapping client library ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: python from ansys.api.geometry.v0.designs_pb2 import ( ExportDesignRequest, NewDesignRequest, SaveAsDocumentRequest, ) from ansys.api.geometry.v0.designs_pb2_grpc import DesignsStub The best practice is to create a Pythonic client library that organizes the service methods in a user-friendly manner. At a minimum, this library should act as a facade layer wrapping the service calls so that the Pythonic API can have a consistent abstraction, independent of underlying implementations. For each client library release, only a single gRPC API version should be wrapped to maintain a consistent API abstraction expectation for the supporting server instances. Public versus private Python API package ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Making the PROTO files for a public or private repository is up to the owner of each repository. In terms of intellectual property (IP) concerns, the PROTO files are typically not an issue because they do not expose any critical service logic or knowledge. In most cases, the APIs being exposed through the PROTO files are already exposed publicly through other mechanisms. Thus, the general recommendation is to make these repositories public as soon as possible. The main reasons for doing so follow: * Private Python package dependencies usually involve workarounds when setting up the workflow. It is best to keep the workflows as standard and simple as possible. That implies making all its dependencies public, including this API Python package. * The API Python package generated eventually must be uploaded to the public PyPI so that it can be consumed by its corresponding Python client library (when it is publicly released). So, if there are no issues with making it public, it is better to do so sooner rather than later. * Once the Python API package is publicly released to PyPI, there is no reason to keep the repository private because all users who consume the Python API package have direct access to the PROTO files that are in the repository. However, before making any repository public in the `Ansys GitHub organization`_, review the `Ansys Open Source Developer's Guide `_ to verify that the repository is compliant with all the needed requirements.