Saltar al contenido principal

Domo University Videos

Domo

Running a DataFlow

Important: When referencing this page outside of Knowledge Base, use this link: http://knowledge.domo.com?cid=dfrun

Intro

When you "run" a DataFlow, you are executing the script for that DataFlow. The first time you run a DataFlow, it generates one or more new DataSets based on the data in your input DataSets, any SQL transforms you have applied, and how you have indicated that you want them to be combined, also using SQL. On subsequent runs, the DataSets are updated to accommodate new changes to the input DataSets themselves as well as any changes you have made to the SQL for the DataFlow.

Notes:
  • You cannot run a DataFlow if you do not have access to any of its component DataSets.
  • If a DataSet fails to execute, it will retry once automatically with no notifications being sent. If the DataFlow continues to fail after the first attempt, notifications will be sent.
  • In previous versions of DataFlows, you could not execute a DataFlow while a previously executed DataFlow was in the indexing state. This is no longer the case. You can now execute multiple DataFlows at once. 

Important: Input DataSets in a DataFlow cannot be restricted by PDP policies—all available rows must pass through the DataFlow. Because of this you must apply PDP policies to the output DataSets generated by a DataFlow.
 

When you build a DataFlow using an input DataSet with PDP policies in place, the DataFlow breaks unless at least one of the following criteria applies:

  • You have an "Admin" security profile.

  • You are the DataSet owner.

  • You are part of the "All Rows" policy. This gives you access to all of the rows in the DataSet.

For more information about using PDP with DataFlows, see PDP and DataFusions/DataFlows

You can run DataFlows one at a time or in bulk.

Running a Single DataFlow

You can run a single DataFlow in any of the following locations:

  • From the DataFlows tab in the Data Center. This is the fastest way to run a DataFlow; however, the DataFlow must have already been created in the Create DataFlow view.

  • From the Details view for the DataFlow. As with the previously mentioned option, the DataFlow must already be created to do this.

  • From the Create/Edit DataFlow view. In this view, you can choose between Save and Save and Run. The Save option only saves changes you have made to the DataFlow and, if you have just created the DataFlow, adds a card to the DataFlows listing in the Data Center page. The Save and Run option performs the same function as Save and also runs the script for the DataFlow.

To run a DataFlow from the Data Center page,

  1. In Domo, click  > Data Center.
    The Data Center opens with the DataSets tab open by default.

  2. Click data_center_dataflows_icon.png in the left-hand navigation pane.
    The DataFlows tab opens.

  3. Locate the DataFlow you want to run.

  4. You can use the filter options to narrow down the DataFlows that appear in the list.

  5. Mouse over the desired DataFlow, click dataset_gear_icon.png, then select Run.

This runs the script for the DataFlow. If this is the first time you have run this DataFlow, DataSets based on the DataFlow are generated and added to the DataSets tab. If the DataSets have already been generated, they are updated according to changes made to the input DataSets and DataFlow SQL.

To run a DataFlow from the Details view,

  1. In Domo, click  > Data Center.

  2. Click DataFlows in the left-hand navigation pane.

  3. Locate the DataFlow you want to run.
    You can use the filter options to narrow down the DataFlows that appear in the list.

  4. Mouse over the desired DataFlow, click , then select Details.

  5. Click  then select Run.

To run a DataFlow from the Create/Edit DataFlow view,

  1. In Domo, click  > Data Center.
    The Data Center opens with the DataSets tab open by default.

  2. (Conditional) Do one of the following:

    • If you have not yet created the DataFlow...

      1. Click New > ETL or New > SQL.

      2. Follow the steps provided in Creating a Magic ETL DataFlow or Creating an SQL DataFlow

      3. Once you have created the DataFlow, click Save and Run.

        This runs the script for the DataFlow. DataSets based on the DataFlow are generated and added to the DataSets tab, and an entry for the DataFlow appears in the DataFlow tab.

    • If the DataFlow already exists...

      1. Click data_center_dataflows_icon.png in the left-hand navigation pane to open the DataFlows tab.

      2. Locate the DataFlow you want to run.
        For more information about searching and filtering DataFlow in the Data Center, see Data Center Layout.

      3. Mouse over the DataFlow row and click the dataset_gear_icon.png icon.

      4. Select Edit.

      5. Make any changes you want (for more specific information, see Creating a Magic ETL DataFlow or Creating an SQL DataFlow) then click Save and Run.

        This runs the script for the DataFlow. All DataSets based on the DataFlow are updated according to changes made to the input DataSets and the DataFlow SQL.

For more information about components of the Data Center and the DataFlows tab, see Data Center Layout.

Note: Many users ask why output DataSets for a DataFlow are not marked as "Updated" when the DataFlow runs successfully. This is usually because the data has not actually changed—no update has occurred. Therefore, the DataSets do not show as updated.

Running Multiple DataFlows 

You can run multiple DataFlows at once by selecting them in the Data Center then choosing the Run Now option.

To run multiple DataSets at once,

  1. Open the Data Center by selecting Data Center in the new_app_icon.png menu.
    The Data Center opens with the DataSets tab open by default.

  2. Click data_center_dataflows_icon.png in the left-hand navigation pane.
    The DataFlows tab opens.

  3. Locate one of the DataFlows you want to run.
    For more information about searching and filtering DataFlow in the Data Center, see Data Center Layout.

  4. Mouse over the row for the DataFlow and click the circled checkmark that pops up over the connector icon. 

    data_center_dataflows_bulk.png

  5. In the blue bar that appears at the top of the screen, click the data_center_bulk_gear_icon.png icon.

  6. Select Run Now.

    data_center_dataflows_bulk_run.png

    All of the DataFlows you have selected should now run.

Cancelling a DataFlow

If you wish to cancel an executed DataFlow mid-run, you may do so by selecting the Cancel option from the  menu. To protect data integrity, DataFlows cannot be cancelled once they reach the "Indexing" stage; they must continue running until completion. 

Training Video - Cancel a DataFlow

 

 

Note: The product training videos are for Domo customers and clients only.