Fork a run

Run forking is in Public Preview for Multi-tenant Cloud and Dedicated Cloud. It is not currently available for Self-Managed deployments.Public Preview features are available for production evaluation, but functionality, APIs, and behavior are in active development and may change before General Availability.

To explore different hyperparameters or models from a specific point in an experiment without impacting the original run, fork from an existing W&B run. When you fork from a run, W&B creates a new run using the source run’s unique ID and a specified step. W&B copies summary metrics from the source run to the forked run. The forked run shares all history and files from the source run up to the specified step. After the fork step, you can log new data to the forked run independently of the original run. This lets you compare alternative training trajectories side by side without rerunning earlier steps or overwriting the original run’s history. View a live demo of a forked run produced by the following code.

Forking a run requires wandb SDK version 0.16.5 or later.
Forking a run requires monotonically increasing steps. You can’t fork from a run that uses non-monotonic steps defined with define_metric(). Non-monotonic steps break the chronological order of run history and system metrics.

Specify the source run’s unique run ID and the step you want to start the forked run from as arguments to fork_from in wandb.init(). The following sections describe two common ways to fork a run: fork from a run that already exists in W&B, and fork from a run that you create in the same script.

Fork from a previously logged run

Use this approach when the source run already exists in W&B (for example, a completed training run whose ID you can look up in the W&B App). The following code snippet shows how to fork from a run that you previously logged to W&B. First, obtain the run ID of the run you want to fork from. Next, specify the run ID and the step you want to fork from as arguments to fork_from in wandb.init(). Copy and paste the following code into a Python script or notebook cell. Replace [SOURCE-RUN-ID], [PROJECT], and [ENTITY] with your own values.

import wandb

# The unique ID of the source run to fork from
source_run_id = "[SOURCE-RUN-ID]"

# Specify the step to fork from
fork_step = 200

# Fork the run
with wandb.init(
    project="[PROJECT]",
    entity="[ENTITY]",
    fork_from=f"{source_run_id}?_step={fork_step}",
) as forked_run:
    pass

Fork from a run in the same script

Use this approach when you want to fork from a run that you just created without looking up the run ID in the W&B App. The following code snippet shows how to create a run and fork from that run within the same script. First, initialize a run and log some data. Next, use the original run object’s id property to obtain the run ID of that run. Finally, initialize a new run and pass the original run’s ID and the step you want to fork from as arguments to fork_from in wandb.init().

import wandb

# Initialize a run
with wandb.init(
    project="[PROJECT]",
    entity="[ENTITY]"
) as original_run:
    # ...training logic goes here ...
    pass

# Specify the step to fork from
fork_step = int("[NUM]")

# Use the original run's ID and specify the step to fork from
with wandb.init(
    project="[PROJECT]",
    entity="[ENTITY]",
    fork_from=f"{original_run.id}?_step={fork_step}",
) as forked_run:
    # ...training logic goes here ...
    pass

Use the original_run.id property to obtain the unique run ID of the original run.

Example script

The following end-to-end example shows how to first fork a run and then log metrics to the forked run starting from a training step of 200. It demonstrates a full workflow that you can run as-is to see forking in action. Copy and paste the following code into a Python script or notebook cell. Replace [PROJECT] and [ENTITY] with your own values.

import wandb
import math

# Initialize the first run and log some metrics
with wandb.init(
    project="[PROJECT]",
    entity="[ENTITY]"
) as run1:
    for i in range(300):
        run1.log({"metric": i})

# Fork from the first run at a specific step and log the
# metric starting from step 200
with wandb.init(
    project="[PROJECT]",
    entity="[ENTITY]",
    fork_from=f"{run1.id}?_step=200"
) as run2:
    # Continue logging in the new run
    # For the first few steps, log the metric as is from run1
    # After step 250, start logging the spikey pattern
    for i in range(200, 300):
        if i < 250:
            # Continue logging from run1 without spikes
            metric_value = i
        else:
            # Introduce the spikey behavior starting from step 250
            metric_value = i + (2 * math.sin(i / 3.0))  # Apply a subtle spikey pattern

        # Log both metrics in a single call to ensure they're
        # logged at the same step
        run2.log({
            "metric": metric_value,
            "additional_metric": i * 1.1
        })

After you run the example, you have two runs in your project: the original run (run1) with the full training history, and the forked run (run2) that branches from step 200 and logs its own diverging metrics from there.

Rewind and forking compatibilityForking complements a rewind by providing more flexibility to manage and experiment with your runs.When you fork from a run, W&B creates a new branch off a run at a specific point so you can try different parameters or models.When you rewind a run, you can correct or modify the run history itself.

Guides

Integrations

Reference

Fork from a previously logged run

Fork from a run in the same script

Example script

​Fork from a previously logged run

​Fork from a run in the same script

​Example script

Fork from a previously logged run

Fork from a run in the same script

Example script