Skip to main content

​
Overview

Vercel AI sdk is a very populate javascript/typescript framework to interact with various LLMs providers. This page shows how to integrate it with ScrapeGraph

Official Vercel AI documentation

View the integration on LlamaHub

​
Installation

Follow out javascript sdk installation steps using your favourite package manager: 
# Using npm
npm i scrapegraph-js

# Using pnpm
pnpm i scrapegraph-js

# Using yarn
yarn add scrapegraph-js

# Using bun
bun add scrapegraph-js
Then, install vercel ai with their openai provider 
# Using npm
npm i ai @ai-sdk/openai

# Using pnpm
pnpm i ai @ai-sdk/openai

# Using yarn
yarn add ai @ai-sdk/openai

# Using bun
bun add ai @ai-sdk/openai

​
Usage

ScrapeGraph sdk can be used like any other tools, see vercel ai tool calling doc 
import { z } from "zod";
import { generateText, tool } from "ai";
import { openai } from "@ai-sdk/openai";
import { smartScraper } from "scrapegraph-js";

const apiKey = process.env.SGAI_APIKEY;

const ArticleSchema = z.object({
  title: z.string().describe("The article title"),
  author: z.string().describe("The author's name"),
  publishDate: z.string().describe("Article publication date"),
  content: z.string().describe("Main article content"),
  category: z.string().describe("Article category"),
});

const ArticlesArraySchema = z
  .array(ArticleSchema)
  .describe("Array of articles");

const result = await generateText({
  model: openai("gpt-4.1-mini"),
  tools: {
    scrape: tool({
      description: "Get articles information for a given url.",
      parameters: z.object({
        url: z.string().describe("The exact url."),
      }),
      execute: async ({ url }) => {
        const response = await smartScraper(
          apiKey,
          url,
          "Extract the article information",
          ArticlesArraySchema
        );
        const result: z.infer<typeof ArticleSchema> = response;
        return result;
      },
    }),
  },
  prompt: "Can you find me the articles on https://scrapegraphai.com/blog?",
});

console.log(result);
TODO ADD THE LOGS

​
Support

Need help with the integration?

GitHub Issues

Report bugs and request features

Discord Community

Get help from our community