Build an advanced multi-level workflow to handle different types of website navigation (standard vs. JavaScript) and merge the results.
item_name
and sugar_grams
from the menu category pages of McDonald’s and Wendy’s. The key challenge is that each site requires a different automation approach:
Open sub-pages
block to navigate the menu, as direct links might not be available.Open datasets
(Input)Layout
menu (Add new row
, Link actions
)Find Links
(Transform)Follow Links
(Input)Open sub-pages
(Transform)Interact with Page
(Transform)Extract Data
(Transform)Extract Data
block.Open datasets
block.URL
and contain
.URL
column will hold the links to the menu category pages.contain
column will hold a text snippet that is unique to the product links on that site, which helps the Find Links
block.Open datasets
block containing the data from the table after.URL | contain |
---|---|
https://www.mcdonalds.com/gb/en-gb/menu/whats-new.html | /product/ |
https://www.mcdonalds.com/gb/en-gb/menu/burgers.html | /product/ |
Find Links
block. Configure it with a dynamic goal: Find all links to each product's page that must contain '{{contain}}'
. This uses the variable from our dataset.Follow Links
block next. Configure it to Follow each link
found by the previous step.Interact with Page
block to click the spoiler. Configure it with the action: Click on "Nutritional Information"
.Layout
menu in the top action bar.Add new row
. A new “start” placeholder will appear on your canvas. Add Open datasets
block from the top menu.Open datasets
block. It will use the same data as the first one.URL |
---|
https://order.wendys.com/category/100/hamburgers?lang=en_US |
https://order.wendys.com/category/149/english-muffins?lang=en_US |
Open sub-pages
block after it. This block will handle the JavaScript-driven menu.
open each menu item on the page
20
.Extract Data
block. This block will have the final context from whichever branch was active for a given input row.
Layout
menu and select Link actions
.Open sub-pages
block (the last block of Row 2), then click on the Extract Data
block, it’s at the end of the first brunch. Now, second branch feed into it.Screenshot: A multi-level workflow showing two parallel rows being linked into a single, final Extract Data block
Extract Data
BlockExtract Data
block.A single item
.NAME | DESCRIPTION |
---|---|
item_name | <the name of the menu item> |
sugar_grams | <the sugar content in grams (g)> |