Explore Content

The Explore Content block gives your AI Agent an intelligent search capability. Instead of providing exact links, you give the agent a goal, and it will perform a series of attempts to navigate the current website and find pages relevant to that goal.

This block is exceptionally useful when you don’t know the exact URLs where information resides but can describe what you’re looking for. With each attempt, the agent learns from the previously visited pages to improve its search.

​

Purpose

Use the Explore Content block when you want your AI Agent to:

• Iteratively find specific types of pages within a website (e.g., “Find the contact page,” “Locate pages about pricing”).
• Discover information that might be spread across several internal pages by learning from each navigation step.
• Intelligently explore a section of a website to find the most relevant content before extracting data.

This block is followed by an Extract Data block, which is used to both analyze the content of each attempt and to extract the final data once the goal is reached.

​

Configuration

​

1. Describe an action or set of actions below…

This is the primary input field where you define the ultimate goal for the AI Agent. The agent will use this goal to evaluate its success after each attempt. Be clear and specific.

• Example: "Find the page that contains the company's contact information, including a physical address."

​

2. How many pages to open?

This parameter sets how many times the agent will try to find the target page. For each new attempt, the agent receives the context of the data that was extracted in the previous attempt and whether that attempt was successful in reaching the goal.

• Example: Setting this to 3 means the agent will make up to three attempts. If it finds a page and extracts data that matches the goal on the first attempt, it will stop. If not, it will use what it learned to make a second, more informed attempt.

​

3. Maximum number of steps

This defines the “budget” of interactions available to the agent for each single attempt. For example, a value of 2 allows the agent to perform up to two interactions to find the page within one attempt.

An interaction is a single action the agent takes to navigate or change the state of the page. This can be a click on a traditional link, or a click on an interactive element like a button, a product card, or a menu item.

​

How It Works

The Explore Content block operates in a loop for the specified number of attempts:

1. Attempt Start: The agent starts on the current page.
2. Navigation: It uses its goal description to decide which interactions to perform. It will perform up to the number of interactions specified in its “interaction budget” for the current attempt.
3. Content Analysis: After each attempt’s navigation path is complete, the subsequent Extract Data block is executed on the final page reached.
4. Goal Evaluation: The agent analyzes the extracted data to determine if the goal has been met.
5. Iteration:
   • If the goal is met, the process stops, and the successful data is passed on.
   • If the goal is not met and there are attempts remaining, the agent starts a new attempt. It uses the information and extracted data from the failed attempt as context to make a smarter choice for its next navigation path.

​

Example: Finding a Specific Case Study on a Landing Page

Imagine you’re on a B2B software company’s homepage, and you want to find a case study about a client in the “financial services” industry. This information might be under “Solutions”, “Customers”, or “Resources”.

1. Open Websites Block:
   • URL: https://some-b2b-software.com
2. Explore Content Block (added next):
   • Describe an action…: Find a case study page for a client in the financial services industry.
   • Number of attempts…: 3
   • How many interactions…: 2
3. Extract Data Block (added next):
   • What information…?: A single item
   • Attributes:

     NAME	EXAMPLE VALUE OR A LONGER DESCRIPTION
     page_title	The main title of the current page.
     mentions_finance	Does the text on this page mention 'finance', 'bank', or 'investment'? Yes or No.

• Result:
  • Attempt 1: The agent might click “Resources” (1st interaction), then “Blog” (2nd interaction). It lands on a blog page. Extract Data runs, and mentions_finance might be “No”. The goal is not met.
  • Attempt 2: Using the context from the failed attempt, the agent knows the blog was a dead end. It starts again from the homepage. This time it might click “Customers” (1st interaction) and see a link for “Case Studies”, clicking it (2nd interaction). It now sees a list of case studies. It might click the first one. Extract Data runs. The result might not mention finance. Goal not met.
  • Attempt 3: The agent now knows it is on the correct “Case Studies” list page. It goes back and intelligently selects a case study with a title like “How Global Bank Optimized Their Workflow”. It navigates to this page. Extract Data runs, and mentions_finance is “Yes”. The goal is met, and the process stops, having successfully found the target page.

​

Key Considerations

• Paired with Extract Data: This block is intrinsically linked to a subsequent Extract Data block, which is used for the critical step of evaluating whether the goal has been achieved on each attempt.
• Clear Goal: The success of this block is highly dependent on a clear and verifiable goal. The agent needs to be able to determine success based on the data extracted.
• Interaction Budget: Use the interaction budget wisely. For finding something that is likely two interactions away (e.g., in a dropdown menu), a budget of 2 is appropriate. For broader exploration, more attempts might be better than a large interaction budget.