Sigma OCF Connector: An Installation and Configuration Guide

Learn how to upload and install Hakkoda's new OCF Connector for Alation, a solution built to facilitate seamless integration between Alation's Data Catalog and Sigma.
September 28, 2023
Share
Hakkoda - Sigma OCF Connector - Thumbnail

Sigma OCF Connector: Overview

Hakkōda developed the OCF connector for Sigma, and it is available as a Zip file that can be uploaded and installed in Manage Connectors. The Sigma OCF connector can be purchased by contacting info@hakkoda.io.

This connector should be used to catalog Sigma as a Business Intelligence source in Alation. The connector catalogs Sigma objects such as Workspaces, Folders, Workbooks, and Pages. Users can find Sigma metadata using the Search features, curate Sigma objects pages, and understand their data lineage from the Alation user interface.

Team

The following administrators are required to install this connector:

  • Alation Server Administrator:
    • Install the OCF Connector
    • Creates a Sigma source
  • Sigma Administrator:
    1. Creates the Sigma API Token

Scope

The table below shows what features are covered by the connector.

FeatureDescriptionAvailability
Authentication
API TokenAuthentication with an API Token created in the Sigma Administration interfaceYes
Extracted Metadata Objects
WorkspacesLogical areas in Sigma where teams can collaborateYes
FoldersOrganizational units within Sigma can contain multiple workbooks or datasets.Yes
WorkbooksInteractive sheets in Sigma where data is analyzed and visualized.Yes
Workbook OwnerThe individual or entity in Sigma who has created or has ownership rights over a specific workbook.Yes
Workbook Created TimestampThe exact time and date when a workbook was initially created in Sigma.Yes
Workbook Updated TimestampThe most recent time and date when a workbook was modified in Sigma.Yes
PagesIndividual screens or tabs in a workbook display specific pieces of analyzed data.Yes
DatasetsCollections of data elements in Sigma that serve as the source for analyses in workbooks including tables, visualizations, etc.Yes
Dataset OwnerThe individual or entity in Sigma who has created or has ownership rights over a specific dataset.Yes
Dataset Created TimestampThe exact time and date when a dataset was initially created in Sigma.Yes
Dataset Updated TimestampThe most recent time and date when a dataset was modified in Sigma.Yes
Alation Catalog Features
Metadata ExtractionThe process of retrieving detailed information about Sigma’s various objects and importing them into Alation.Yes
LineageAbility to generate lineage against the Sigma artifacts

Yes*

* Lineage is only currently available via manual lineage creation. Automated lineage is presently being developed and scheduled for release in Q1 2024.

Sigma Objects Hierarchy

The following diagram shows the hierarchy of Sigma objects and how they are cataloged in Alation:

Hakkoda - Sigma OCF Connector - Mapping

Sigma APIs

The following table lists the Sigma APIs used by this connector to extract metadata from Sigma:

Verb

Rest API Endpoint

Description

Post

{apiURL}/v2/auth/token

Returns the access and refresh tokens

Get

{apiURL}/v2/workspaces

Returns a list of workspaces

Get

{apiURL}/v2/members

Returns a list of members

Get

{apiURL}/v2/members/{memberId}/files

Returns all documents and folders accessible to a member

Get

{apiURL}/v2/workbooks/{workbookId}/pages

Returns a list of pages in a workbook

Get

{apiURL}/v2/workbooks/{workbookId}/pages/{pageId}/elements

Returns a list of elements in a workbook page

Get

{apiURL}/v2/workbooks/{workbookId}/lineage/elements/{elementId}

List lineage dependencies of a workbook element

Get

{apiURL}/v2/workbooks/{workbookId}/elements/{elementId}/columns

Returns the columns for an element in a workbook

The {apiURL} used must correspond to the cloud provider your Sigma organization is hosted on.

For additional information about these APIs, please review the Sigma documentation here.

Sigma OCF Connector: Install and Configure

Follow these steps to perform the Sigma connector configuration within Sigma and Alation.

 

Prerequisites

 

Firewall Configuration:

This section only applies to Alation On-Premise customers.

Ensure the server where your Alation Connection Manager is installed can reach your respective Sigma API URL over port 443.

 

Alation Connection Manager:

This section only applies to Alation On-Premise customers.

If you still need to do so in your instance, install Alation Connector Manager using the steps in Install Alation Connector Manager.

 

Sigma Cloud Provider Determination:

You must determine which cloud provider your instance of Sigma is running on. This will determine which Sigma API URL to utilize in the Connector Configuration.

This information is listed in the Administration Portal under Account > General Settings > Site > Cloud.

Hakkoda - Sigma OCF Connector - Provider

The Sigma Administrator should provide the cloud provider to the Alation Administrator.

Sigma OCF Connector and License Key:

After purchasing the connector, your designated recipient will receive the Sigma OCF Connector zip file and the license key to activate your solution.

I. Sigma: API Token Generation

A Sigma Administrator should perform the following steps.

  1. Open your Sigma Admin Portal.
  2. On the left side of the page, click API & Embed Secrets to open the API & Embed Secrets page.
  3. On the API & Embed Secrets page, click the Create New button.
Hakkoda - Sigma OCF Connector - API Token 1

This will open the Create New API Token or Embed Secret modal.
The API Token option will be selected by default.

Hakkoda - Sigma OCF Connector - API Token 2
  1. Enter a Name to identify the specific API Token. Example: Alation OCF API Token
Hakkoda - Sigma OCF Connector - API Token 3
  1. [optional] Enter a description.
  2. Under Owner, select the user account you want to associate with the token.
    Note: Account type permissions assigned to the selected Sigma user will automatically carry over into the Sigma API Token. The Sigma OCF Connector requires choosing a user with Admin privileges.
Hakkoda - Sigma OCF Connector - API Token 4
  1. Click Create.
  2. An API Key Secret modal will appear. Click Copy and immediately paste the secret in a secure location. The Client Secret is required when configuring the Sigma OCF Connector.
    Important: The secret cannot be retrieved once this modal is closed.
Hakkoda - Sigma OCF Connector - API Token 5
  1. Click Close.
  2. Your newly created Sigma API Token will be listed in the APIs & Embed Secrets panel. Hover over the token’s Client ID and click copy. The Client ID is required when configuring the Sigma OCF Connector.
Hakkoda - Sigma OCF Connector - API Token 6

II. Sigma OCF Connector: Installation

An Alation Server Administrator should perform the following steps.

  1. Ensure that the Sigma OCF connector Zip file you received from Hakkōda is available on your local machine. Note: the name of the zip file should be hakkoda-sigma-ocf_connector.zip
  2. Install the connector on the Connectors Dashboard page: refer to Manage Connector Dashboard.

III. Sigma OCF Connector: Configuration

An Alation Source Administrator or above should perform the following steps.

Add a new BI Server Source

  1. Log in to the Alation instance and add a new BI Server source: Apps > Sources > Add > BI Server. The Register a Business Intelligence Server screen will open.
  2. On this screen, from the Select a Business Intelligence Server type list, select the Sigma – OCF Connector by Hakkoda connector name.
Hakkoda - Sigma OCF Connector - BI Source
  1. Provide the Title and a Description (optional) for your Sigma data source.
Hakkoda - Sigma OCF Connector - BI Source 2
  1. Click the Add button.

 

Configuration in Alation

  Note

This section describes configuring settings for credentials and connection information stored in the Alation database. If your organization has configured Azure KeyVault or AWS Secrets Manager to hold such information, the user interface for the General Settings page will change to include the following icons to the right of most options:

Hakkoda - Sigma OCF Connector - Icons

By default, the database icon is selected, as shown. In the vault case, instead of the actual credential information, you enter the ID of the secret. See Configure Secrets for OCF Connector Settings for details.

Perform the configuration on the Settings page:

  1. Specify the Application Settings:
Hakkoda - Sigma OCF Connector - Application Settings
ParameterDescription
Enable Raw Dump or Replay

The options in this drop list can dump the extracted metadata into files to debug extraction issues before ingesting the metadata into Alation. This feature can be used during testing if there are issues with MDE. It breaks extraction into two steps: first, the extracted metadata is dumped into files and can be viewed, and second, it can be ingested from the files into Alation. It is recommended to keep this feature enabled only if debugging is required.

  • Enable Raw Metadata Dump: Select this option to save extracted metadata into a folder for debugging purposes. The dumped data will be saved in four files (attribute.dump, function.dump, schema.dump, table.dump) in folder opt/alation/site/tmp/ inside Alation shell.
  • Enable Ingestion Replay: Select this option to ingest the metadata from the dump files into Alation.
  • Off: Disable the Raw Metadata Dump or Replay feature. Extracted metadata will be ingested into Alation.
Disable Automatic Lineage Generation

Select the Disable Automatic Lineage Generation checkbox to skip the creation of automatic Lineage after extraction. When automatic Lineage generation is disabled, Alation does not calculate Lineage data for this BI source during extraction.

For more information, see Disable Automatic Lineage Generation.

Disable Permission Enforcement

Leave this checkbox selected.

Permission enforcement is not supported between Sigma and Alation

Server URI

Enter the server URI used to access Sigma. This should be your organization context root: https://app.sigmacomputing.com/<org>/

Example:

https://app.sigmacomputing.com/hakkoda/

Note: The URL must include the final forward slash. In addition, this URL is validated by the connector license key validation step each time the connector runs.

  1. Click Save to save the information you have entered.
  2. Specify Connector Settings:
Hakkoda - Sigma OCF Connector - Connector Settings
Parameter Description
Sigma Web URL Specify the URL to access Sigma. This should match the Server URI parameter value entered in the Application Settings.
Sigma API URL Specify the API URL to access Sigma APIs. The Sigma API URL used is specific to the cloud provider your Sigma instance is running on. Your Sigma Administrator should provide the cloud provider. See Sigma Cloud Provider Determination for the details. After the cloud provider is determined, use the following to determine your specific Sigma API URL:
Sigma Client ID Specify the Sigma client ID. See Sigma: API Token Generation for more details.
Sigma Client Secret Specify the Sigma client secret. See Sigma: API Token Generation for more details.
Sigma OCF License Key Specify the Sigma OCF License Key provided by Hakkōda. See Sigma OCF Connector and License Key for more details.
Proxy URL Specify the URL to access Sigma via a proxy server. This optional field should be used only if the Sigma OCF connection occurs over a proxy. This field should be left blank for Alation Cloud Services customers.
Proxy Port Specify the proxy port number. This optional field should be used only if the Sigma OCF connection occurs over a proxy. This field should be left blank for Alation Cloud Services customers.
Proxy Username Specify the proxy server username. This optional field should be used only if the Sigma OCF connection occurs over a proxy. This field should be left blank for Alation Cloud Services customers.
Proxy Password Specify the proxy server password. This optional field should be used only if the Sigma OCF connection occurs over a proxy. This field should be left blank for Alation Cloud Services customers.
Log Level Select the log level to generate logs. The available options are INFO, DEBUG, WARN, TRACE, ERROR, FATAL, ALL.
  1. Click Save to save the information you have entered.
  2. Under Test Connection, click Test to validate network connectivity.
Hakkoda - Sigma OCF Connector - Connection
  1. Under Extraction Settings, turn on Selective Extraction, if required. Selective extraction settings apply a filter to include or exclude a list of Sigma Workspaces.
    1. Click Get List of Sigma Workspaces to fetch the list of Sigma Workspaces.
    2. The status of the Get List of Sigma Workspaces action is logged in the Job History table at the bottom of the Settings page.
    3. Once the Sigma Workspace synchronization succeeds, a drop-down list of Sigma Workspaces will be enabled. 
    4. Select one or more Sigma Workspaces to apply the filter.
    5. Check if you are using the desired filter option. Available filter options are described below:
Filter OptionDescription
Extract all Sigma Workspaces exceptExtract metadata from all projects except from the Sigma Workspaces selected.
Extract only these Sigma WorkspacesExtract metadata from only the selected Sigma Workspaces.
  1. Click Run Extraction Now to extract metadata. The status of the extraction action is also logged in the Job History table at the bottom of the page.
    1. If you wish to automatically update the metadata in the Catalog, under Automated and Manual Extraction, turn on Enable Automated Extraction and select the day and time when metadata must be extracted. The metadata extraction will be automatically scheduled for the selected day and time.

Troubleshooting

For general connector issues, refer to Troubleshooting.

FSI State of Data: Financial Services and Insurance Orgs are Falling Behind in Data Modernization. Here’s How They Can Catch Up.

FSI State of Data: Financial...

Hakkoda’s Financial Services and Insurance State of Data Report reveals key gaps in data strategy, barriers to data modernization, and…
Hakkōda 2024 Financial Services & Insurance State of Data Report: 97% Say Generative AI Matters for Success

Hakkōda 2024 Financial Services &...

Hakkoda releases its 2024 Financial Services & Insurance State of Data Report, surveying 145 director to CEO level data leaders…
Rainfall MDM

Rainfall MDM

The Sigma OCF Connector facilitates seamless integration between Alation's Data Catalog and Sigma, enabling comprehensive data discovery, governance, and search…

Never miss an update​

Join our mailing list to stay updated with everything Hakkoda.

Ready to learn more?

Speak with one of our experts.