Getting started with Flyte SDK for Python
This guide will help you get started with flytekit, the Python SDK for Flyte. Flyte is a workflow automation platform that helps you build reliable, scalable, and maintainable data and ML workflows.
Prerequisites
Before you begin, ensure you have the following installed:
- Python:
flytekitrequires Python versions>=3.9,<3.13. We recommend using Python 3.9, 3.10, 3.11, or 3.12. - Operating System:
flytekitis primarily supported on Linux. - System Dependencies: You might need
build-essential,vim,libmagic1, andgitfor certain functionalities or if building from source.
Install
We recommend using uv for a fast and efficient installation.
-
Install
uv: If you don't haveuvinstalled, you can get it viapip:pip install uv -
Create a Virtual Environment: It's good practice to install Python packages in a virtual environment to avoid conflicts with system-wide packages.
python -m venv .venv
source .venv/bin/activate -
Install Flytekit: Use
uvto installflytekit. You can also include optional dependencies likeagentorconnectorif needed.uv pip install flytekit[agent,connector]
Hello World
Let's create a simple Flyte workflow and run it locally.
-
Create a Python file (e.g.,
my_flyte_app.py):# my_flyte_app.py
from flytekit import task, workflow
@task
def greet(name: str) -> str:
"""A simple task that returns a greeting."""
return f"Hello, {name}!"
@workflow
def my_workflow(name: str) -> str:
"""A workflow that calls the greet task."""
greeting = greet(name=name)
return greeting
if __name__ == "__main__":
# Run the workflow locally using Python directly
print(f"Running my_workflow locally: {my_workflow(name='Flyte User')}")
# You can also run it using the pyflyte CLI for a more complete local simulation
# To do this, save the file and then run from your terminal:
# pyflyte run my_flyte_app.py my_workflow --name "Flyte CLI User" -
Run the workflow locally:
python my_flyte_app.pyYou should see output similar to:
Running my_workflow locally: Hello, Flyte User!
Verify Installation
To verify that flytekit and its CLI tools are correctly installed, check the version of pyflyte:
pyflyte --version
You should see the installed flytekit version.
Other Install Options
Using pip
If you prefer pip over uv, you can install flytekit as follows:
python -m venv .venv
source .venv/bin/activate
pip install flytekit[agent,connector]
Using Docker
You can also use Docker to run flytekit in a containerized environment. The project provides Dockerfile examples.
To build a Docker image:
docker build -f Dockerfile.dev --build-arg PYTHON_VERSION=3.10 -t localhost:30000/flytekittest:someversion .
Then you can run your Flyte workflows within this image. For example, to run a workflow with a specific image:
pyflyte run --image localhost:30000/flytekittest:someversion my_flyte_app.py my_workflow --name "Docker User"
Troubleshooting
- Python Version Mismatch: If you encounter issues, ensure your active Python environment matches the required versions (
>=3.9,<3.13). You can check your Python version withpython --version. - Missing Dependencies: If you see
ModuleNotFoundErroror similar, ensure all necessary dependencies are installed. Re-runuv pip install flytekit[agent,connector]orpip install flytekit[agent,connector]to catch any missing packages. - Rich Tracebacks: If you need more detailed error messages for debugging, you can enable rich tracebacks by setting an environment variable:
export FLYTE_SDK_RICH_TRACEBACKS=1 PYTHONPATHIssues: Ifpyflytecommands or imports fail, check yourPYTHONPATHenvironment variable. It might need to include your project root or theflytekitinstallation path.echo $PYTHONPATH
# Example: export PYTHONPATH="/path/to/your/project:$PYTHONPATH"
Next Steps
Now that you have flytekit installed and can run a basic workflow, explore the official Flyte documentation and examples to learn more about:
- Defining more complex tasks and workflows.
- Running workflows on a remote Flyte cluster.
- Using advanced features like caching, scheduling, and data types.
- Interacting with the Flyte CLI (
pyflyteandflyte-cli).