Zapier & Make HTML Webscraping Parser

Zapier and Make (Integromat) provide a great no-code platform to build simple workflows. However, sometimes you need to add additional functionality that's not available inside the platform natively.

Autocode allows you to create custom endpoints that can be called from inside Zapier and Make. This App is an example of how you can use Autocode endpoints inside Make and Zapier. It provides you with three HTML parsers:

functions/emails.js - which extracts all emails out of a given webpage
functions/phonenumbers.js - which extracts all phone numbers out of a given webpage
functions/html.js - which is a more general HTML parser, and allows you to convert the contents of specified HTML elements to JSON

Follow the guide below to learn how to setup this App for use in Zapier or Make!

Setup

You will need

A free Autocode account
Either a Make (Integromat) or Zapier account
- We'll explain how to connect this API to both later in this guide

Installing this Autocode App

Scroll up on this page and click the green Install Free button.

Select the organization you'd like to install this app into, give your project a name, then click Install App.

Awesome - your Autocode app is all setup and ready to go! To find the endpoint you'll need to use within Make or Zapier, open the parser you'd like to use (functions/emails.js, functions/phonenumbers.js, or functions/html.js) from the sidebar. You can then find your endpoint's URL at the bottom of the editor. Copy the URL of the paser you'd like to use - you'll need it shortly!

Using the parser on Make

Create a new Make project or open an existing one. Then, click the purple plus icon, type http in the search box, then click the HTTP option that appears.

In the popup that appears, click the Make a request option.

Time to setup our request to our parser in Make!
- In the URL input, paste the URL of the parser you'd like to use. You copied this earlier in this tutorial.
- In the Method input, select GET
- Click the Add parameter button, then:
  - In the Name input that appears, enter url
  - In the Value input that appears, enter the URL that you'd like to parse data from
  - If you're using the functions/html.js parser to parse HTML, you'll need to click Add parameter again. You do not need to do this step when using the functions/emails.js and functions/phonenumbers.js parsers.
    - In the second Name input, enter selector
    - In the second Value input, enter the CSS selector you'd like to use to get data. For example, if you'd like to get the contents of all the <h2> elements on a page, you'd enter h2 into this box. Likewise, if you'd like to get the contents of all elements with the item class, you'd enter .item.
- Set the Parse response option to Yes
Then click OK.

Okay, now we need Make to know what our data looks like. Click the Run Once button, then wait for the the scenario to finish running.

Our parser returns a JSON object that could contain multiple results. We'll use an interator so that we can use this data inside Make. Click the green cog item at the bottom of the editor, then click Iterator.

Click the array input, then click the phonenumbers[], emails[] or data[] button - depending on which parser you're using. If you don't see any of these options, click the small down arrow next to the Data button. Once your screen looks similar to the screenshot below, click OK.

Awesome - your parser and itererator is now setup on Make! You can now connect whichever Make connection (API) you'd like to use your data with to your Make scenario.

Using the parser on Zapier

Please note: using custom webhooks requries a paid Zapier account.

Create a new Zap in Zapier, and setup a trigger for it. In the example below, we're using a scheduled trigger, but you can use any. Keep pressing Continue or Test until your trigger is totally setup.

For your action, select Webhooks by Zapier.

Click the Event dropdown and select GET. Then click Continue.

Time to setup our request to our parser in Zapier!
- In the URL input, paste the URL of the parser you'd like to use. You copied this earlier in this tutorial.
- In the Query String Params section:
  - In the small input on the left, enter url
  - In the larger input on the right, enter the URL that you'd like to parse data from
  - If you're using the functions/html.js parser to parse HTML, you'll need to click + button to add a second parameter. You do not need to do this step when using the functions/emails.js and functions/phonenumbers.js parsers.
    - In the second small input on the left, enter selector
    - In the second larger input on the right, enter the CSS selector you'd like to use to get data. For example, if you'd like to get the contents of all the <h2> elements on a page, you'd enter h2 into this box. Likewise, if you'd like to get the contents of all elements with the item class, you'd enter .item.

Then click Continue.

Click the Test and Continue button that appears.

Awesome - your parsed data is now accessible inside Zapier! You can now add more actions to your Zap and use the parsed data within those actions.