Skip to main content

Flyte Decks for Enhanced Task Visibility

Flyte Decks provide a powerful mechanism to enhance task visibility by generating rich, customizable HTML reports during task execution. They allow developers to gain deeper insights into task inputs, outputs, intermediate states, and performance characteristics directly within the Flyte UI.

Core Concepts and Capabilities​

The Deck Class​

The Deck class is the fundamental building block for creating these reports. It acts as a container for HTML content, which is generated by various renderers.

  • Each Deck instance is identified by a name and accumulates HTML content via its append method. The append method takes an HTML string and concatenates it to the deck's existing content.
  • Flyte automatically manages a list of Deck objects within the task's execution context. When a task completes, the HTML content from all registered decks is rendered into a comprehensive report.
  • Renderers: Deck objects leverage renderers (e.g., FrameRenderer for DataFrames, MarkdownRenderer for Markdown text) to convert structured data or strings into HTML. This modular approach allows for flexible content generation.
  • Automatic Deck Management: By default, Deck instances are automatically added to the current Flyte context upon initialization, simplifying their integration. This behavior can be controlled with the auto_add_to_deck parameter in the Deck constructor.
  • Publishing Decks: The static method Deck.publish() is responsible for triggering the rendering and output of deck content. This is typically handled automatically by Flyte. Deck generation can be disabled globally via the enable_deck flag in the Flyte context, in which case publish() issues a warning and returns.

Default Decks​

Every Flyte task automatically comes with several pre-configured decks to provide immediate visibility:

  • Input Deck: Renders the task's input data.
  • Output Deck: Renders the task's output data.
  • Default Deck: A general-purpose deck for displaying plots, charts, or arbitrary Markdown content.

Developers can access the default_deck via flytekit.current_context().default_deck to append content.

Custom Decks​

Beyond the default decks, developers can instantiate new Deck objects with unique names to organize and present specific information. This allows for highly tailored reports.

Overriding Default Renderers​

For task inputs and outputs, the default rendering behavior can be overridden using Annotated type hints. By associating a specific renderer with a type hint, developers can control how their data is visualized in the input/output decks.

Specialized Decks​

  • TimeLineDeck: This specialized Deck subclass is designed for profiling task execution. It collects timing information for different parts of a task and renders it as a timeline graph. Unlike the base Deck, TimeLineDeck delays HTML generation until all timing data is collected, ensuring a complete and meaningful visualization. Timing information is appended using the append_time_info method, which accepts a dictionary containing details like name, start, and end times for a segment.
  • DeckField: This enumeration (INPUT, OUTPUT, SOURCE_CODE, TIMELINE, DEPENDENCIES) categorizes the types of information that can be displayed within a deck, indicating potential built-in or standard report sections.

Common Use Cases and Examples​

Visualizing DataFrames​

Displaying the head or tail of a Pandas DataFrame as an HTML table in the output deck.

import pandas as pd
import flytekit
from flytekit import task
from flytekit.deck import TopFrameRenderer
from typing_extensions import Annotated

@task
def process_data() -> Annotated[pd.DataFrame, TopFrameRenderer(10)]:
    """
    Returns a DataFrame, with only the first 10 rows rendered in the output deck.
    """
    df = pd.DataFrame({"col1": range(100), "col2": [f"item_{i}" for i in range(100)]})
    return df

Adding Custom Markdown Reports​

Including descriptive text, summaries, or links in a task's report, either to the default deck or a custom deck.

import flytekit
from flytekit import task
from flytekit.deck import MarkdownRenderer

@task
def generate_report():
    """
    Generates a task summary and detailed metrics using Markdown.
    """
    md_text = """
    # Task Summary
    This task processed **100 records**.
    - **Status:** Completed Successfully
    - **Key Metric:** Accuracy = 0.95
    """
    # Append to the default deck
    flytekit.current_context().default_deck.append(MarkdownRenderer().to_html(md_text))

    # Create a custom deck for specific details
    custom_deck = flytekit.Deck("Detailed Metrics")
    custom_deck.append(MarkdownRenderer().to_html("## Detailed Analysis\n- Precision: 0.92\n- Recall: 0.98"))

Plotting Data​

Embedding interactive plots (e.g., using Plotly, Matplotlib rendered to HTML) directly into the task report.

import flytekit
from flytekit import task
import plotly.express as px
import pandas as pd

@task
def plot_iris_data():
    """
    Generates and embeds a Plotly scatter plot of the Iris dataset.
    """
    iris_df = px.data.iris()
    fig = px.scatter(iris_df, x="sepal_width", y="sepal_length", color="species")
    # Append the plot's HTML representation to the default deck
    # full_html=False and include_plotlyjs='cdn' are important for embedding.
    flytekit.current_context().default_deck.append(fig.to_html(full_html=False, include_plotlyjs='cdn'))

Profiling Task Execution​

Using TimeLineDeck to visualize the duration of different code segments within a task.

import flytekit
from flytekit import task
from flytekit.deck import TimeLineDeck
import time

@task
def profile_task_segments():
    """
    Profiles different segments of a task's execution using TimeLineDeck.
    """
    timeline_deck = TimeLineDeck("Execution Profile")

    start_time = time.time()
    # Simulate data loading
    time.sleep(0.1)
    end_time = time.time()
    timeline_deck.append_time_info({"name": "Data Loading", "start": start_time, "end": end_time})

    start_time = time.time()
    # Simulate model inference
    time.sleep(0.2)
    end_time = time.time()
    timeline_deck.append_time_info({"name": "Model Inference", "start": start_time, "end": end_time})

    # The TimeLineDeck automatically renders this information when its html property is accessed by Flyte.
    # Note: For accurate measurements, refer to wall time and process time metrics.

Best Practices and Considerations​

  • Keep Decks Concise: While powerful, avoid generating excessively large HTML reports, as this can impact UI performance and storage. Focus on key insights and summaries.
  • Leverage Annotated: For standard input/output visualization, Annotated provides a clean and declarative way to specify renderers directly in your task signatures.
  • Modular Renderers: Consider creating custom renderers for complex data types or specific visualization needs to maintain clean and reusable code.
  • Conditional Deck Generation: Use the enable_deck flag in the Flyte context to control when decks are generated, especially in production environments where detailed reports might not always be necessary.
  • Security: Be mindful of the content rendered in decks, especially if it includes user-generated or sensitive information, as it will be displayed in the Flyte UI. Ensure proper sanitization if displaying untrusted input.