Skip to main content
Version: v3.0.0

Joined feature sets example

import h2o_featurestore
from h2o_featurestore import CSVFile, SnowflakeTable, JoinFeatureSets, JobType

# Initialise feature store client
client = h2o_featurestore.login()

# Set workspace specifics
workspace = client.workspaces.create("demo")

# Create first feature set
csv = CSVFile("<path to csv file>")
csv_schema = workspace.extract_schema_from_source(csv)
fs_1 = workspace.feature_sets.register(csv_schema, "feature_set_1", primary_key=["key"])
fs_1.ingest(csv)

# Create second feature set
snowflake_table = SnowflakeTable(url="<snowflake account url>", warehouse="warehouse name", database="database name", schema="schema name", table="table name")
snowflake_table_schema = workspace.extract_schema_from_source(snowflake_table)
fs_2 = workspace.feature_sets.register(snowflake_table_schema, "feature_set_2", primary_key=["key"])
fs_2.ingest(snowflake_table)

# Create joined feature set transformation
join_transformation = JoinFeatureSets(left_key = "key", right_key = "key")
input_schema = workspace.extract_derived_schema([fs_1, fs_2], join_transformation)
joined_fs = workspace.feature_sets.register(input_schema, "joined_feature_set")

# Get ingest job
auto_ingest_job = joined_fs.get_active_jobs(JobType.INGEST)[0]
auto_ingest_job.wait_for_result()

# Retrieve feature set
ref = joined_fs.retrieve()
ref.download()

Joining feature sets from different workspaces​

The parent feature sets of a join do not have to live in the same workspace. The workspace you call extract_derived_schema on is the one that runs the extraction job and owns the resulting feature set; the parents may come from anywhere you have access to.

import h2o_featurestore
from h2o_featurestore import JoinFeatureSets, JobType

client = h2o_featurestore.login()

# The workspace that will own the joined feature set
target_workspace = client.workspaces.list(name="analytics")[0]

# Parent feature sets from two different workspaces
customers_ws = client.workspaces.list(name="customers")[0]
fs_1 = customers_ws.feature_sets.get_by_name("customer_profiles")

transactions_ws = client.workspaces.list(name="transactions")[0]
fs_2 = transactions_ws.feature_sets.get_by_name("transaction_history")

# Extraction and registration are scoped to the target workspace
join_transformation = JoinFeatureSets(left_key="customer_id", right_key="customer_id")
input_schema = target_workspace.extract_derived_schema([fs_1, fs_2], join_transformation)
joined_fs = target_workspace.feature_sets.register(input_schema, "customer_360")

auto_ingest_job = joined_fs.get_active_jobs(JobType.INGEST)[0]
auto_ingest_job.wait_for_result()
note

A cross-workspace join requires:

  • Editor permission on every parent feature set, in whichever workspace it lives. Read access is not sufficient.
  • Permission to register feature sets in the target workspace, which the editor role on that workspace grants.

Each parent feature set must also have been ingested at least once before it can be used as a join input.

If you do not know which workspace a feature set lives in, search across all of them with client.workspaces.list_feature_sets().

Feedback